‘Big data’ is a term we use to describe analyzing masses of information to reveal answers to questions that would otherwise go unanswered. It’s the science of finding needles in haystacks.

Big data can illuminate hidden trends and relationships, improving understanding, predictions, and decision making. It’s become a major driver of change in every industry imaginable, and is now an invisible yet invaluable companion in everyday life.

From your morning weather forecast, to the app that finds you the fastest travel route, to your latest streaming recommendation — all are products of big data.

On the broader scale, energy providers are using big data to predict demand [1], investment firms are using it to get a read on the market [2], and researchers are using it to better understand the spread of diseases [3]. But these examples offer just a glimpse of its future potential.

Big data isn’t without challenges. One being the burden of storing it. What’s more, big data typically grows in both volume and complexity, changing constantly.

Then there’s organizing, structuring, and actually analyzing it — which is where artificial intelligence (AI) can be an indispensable tool. The biggest challenge of big data, however, is sourcing it. To conduct analyses and arrive at actionable insights, you need high-quality data to begin with.

Big data is entwined in everything we do, supporting our R&D efforts across healthcare, life sciences, and electronics. Elsewhere, we’re using big data to ensure our chemical products are pure [4], and in creating a smart supply chain so patients can access medicines when and where they need them [5].

But we’re also looking outward to help others unlock the possibilities of big data. Alongside Project Data Sphere – an initiative of the nonprofit CEO Roundtable on Cancer, of which we’re a member – we spearheaded the Global Oncology Data Alliance. This ongoing program makes huge volumes of oncology data accessible to researchers worldwide [6, 7].

More recently, we’ve established two partnerships with Palantir, a company that builds software enabling data-driven decision making across multiple industries. These subsidiaries, Syntropy™ and Athinia™, combine the data integration and analytics capabilities of Palantir’s Foundry platform with our deep expertise in healthcare and semiconductors. And through these partnerships, we’re overcoming both common and unique barriers to big data collaboration to transform these industries for the better.

Healthcare data is a vital resource in advancing the understanding and treatment of diseases. And there’s no shortage of it. From test results and X-rays to lab research and clinical trials, massive amounts of data are being created every day.

By 2025, it’s estimated that the compound annual growth rate of healthcare data will reach 36% [8]. Having so much to work with is an opportunity and a challenge. On average, data scientists spend 80% of their time simply preparing data for analysis, and only 20% actually analyzing it [9, 10].

The reason is that different sources of data usually exist within separate systems, or ‘silos.’ Medical records over here and diagnostics over there, as a simple example. Integrating data from different places and in different formats to create one workable data set is extremely time-consuming, taking anywhere from weeks to months. With Syntropy™, it can be done in a fraction of the time.

Syntropy™ sits as a new layer on top of these silos, merging sources together to create structured, analysis-ready assets in one user-friendly system. And rather than being a snapshot in time, the platform is live. Data is always being updated, and new sources can be integrated at any point.

Syntropy™ helps clinicians and researchers spend their time asking questions of big data, rather than preparing it. And a unified platform has another advantage — it sows the seed of collaboration.

Even within a single clinical care center or research institution, individuals, teams, and departments run into barriers that prevent them from working together. Security, compliance, and compatibility issues hinder collaboration. Data sets may contain sensitive information. Researchers may be concerned about how their data will be used, and whether they’ll be credited for their work, which is important for recognition and funding.

Syntropy™ doesn’t take ownership of data. Instead, originators remain in full control. The creator of a data set can allow collaborators to access only certain aspects of it, and specify how it can be used. And no matter what analytics form on top of that data, the creator will forever be linked to it and credited for their contribution within the platform.

Syntropy™ provides a secure environment, built on existing legal and contractual foundations, to enable the entire healthcare industry – from academia to hospitals to pharmaceutical companies – to cooperate in unlocking the potential of big data.

Platform partners already include several leading US cancer centers [11, 12], and by making data easier to work with and creating a space for collaboration, Syntropy™ is accelerating research. This momentum is leading to faster breakthroughs and ultimately, the development of new treatments to improve human lives.

The healthcare and semiconductor industries may seem worlds apart, but the potential of big data has yet to be fully explored in either. What the fields share is opportunities to use big data to chart new paths to progress.

Semiconductor fabrication is incredibly delicate, can take many months, and involves thousands of steps. Intricate chemistry meets precision processing, and any slight impurity or miscalibration along the way can mean failure.

Our expectations of semiconductors are ever increasing, too. Smaller, faster, more efficient microchips are not just desired, but necessary for technological progress. Manufacturing is becoming more complex as a result.

By comparison, data standards have remained somewhat simplistic. Industry certifications only scratch the surface of which parameters can affect the quality and performance of end products. Trial and error development, and quality deviations, are still commonplace.

Manufacturers demand high-quality materials and generate data to monitor and improve processes. But it can take a lone company up to a decade to capture enough data to begin to see where improvements could be made.

Far from discouraged, however, the semiconductor industry is responsible for an explosion of data. This is where Athinia™ comes in, helping companies utilize big data and AI analytics to reveal previously hidden opportunities to optimize manufacturing.

And in creating new data standards using normalization and coding, Athinia™ enables smooth and secure data sharing regardless of the systems that capture it. In such a competitive industry, data sharing may seem risky. However, Athinia™ allows companies to set rigid permissions on what data they make available, with full visibility and control over its use.

Companies can therefore protect their intellectual property in a secure environment, opening up opportunities for collaboration. The vision is to establish an ecosystem where materials suppliers, device makers, and other ecosystem stakeholders collaborate on big data, making it a force for change.

A continuous feedback loop between suppliers and device makers, combined with analytics using AI and machine learning, allows both to be more proactive in identifying the most important parameters and preventing quality deviations before they occur. This predictive capacity sets the stage for better, more transparent partnerships across the industry.

The Athinia™ platform is a game-changer for semiconductor manufacturing efficiency, and this has many knock-on effects. It means fewer stages of testing and optimization, a quicker time to market, and less waste from production failures. This not only helps with sustainability, but knowledge sharing can speed up R&D and enable faster technological innovation.

With pressure on manufacturers at its peak given the ongoing chip shortage that began in 2020 [13], the efficiencies Athinia™ enables have never been needed more.

For many tech firms, data has become a currency. Headline-making data breaches have led to an atmosphere of mistrust around data handling. Whether it’s overlooking patient consent when it comes to the use of their data, or failing to protect a company’s intellectual property, businesses today need to act thoughtfully and responsibly.

We’re taking a different approach with Syntropy™ and Athinia™, which have been designed to offer the highest levels of security, and to spur curiosity without ever taking ownership away from data creators.

But we must also ensure data is used responsibly. This starts with robust anonymization tools that both platforms benefit from. They enable patient and commercially sensitive data to be kept private.

We also formed the Digital Ethics Advisory Panel in 2021 to be at the forefront of ethical digital transformation [14]. This committee comprises independent experts who help us answer the most pressing ethical questions around big data, AI, and other new technologies.

This quickly led to the development of a Code of Digital Ethics that establishes an actionable set of standards with which to approach these ethical questions [15]. Only by putting in place rigid frameworks that prize the responsible use and protection of big data can industries realize its true power. And in doing so, we can collaborate responsibly, and change entire industries. Whether that be accelerating innovation in semiconductors to meet the needs of consumers and new fields like AI and autonomous vehicles; or improving patient care by accelerating research into life-threatening diseases.

^{Additional sources:}

^{Athinia.com
Syntropy.com
projectdatasphere.org
palantir.com}

^{[1] https://www.globenewswire.com/news-release/2022/05/03/2434412/0/en/Big-Data-Analytics-Market-in-the-Energy-Sector-Growth-Trends-COVID-19-Impact-and-Forecasts-2022-2027.html
[2] https://www.smartdatacollective.com/big-data-analytics-has-potential-to-massively-disrupt-stock-market/
[3] https://link.springer.com/article/10.1007/s42979-021-00923-y
[4] https://www.emdgroup.com/en/the-future-transformation/smart-factory.html
[5] https://www.emdgroup.com/en/research/science-space/envisioning-tomorrow/precision-medicine/ai-in-supply-chain.html
[6] https://www.emdgroup.com/en/news/gobda-mou-signing-2017-09-11.html
[7] https://www.emdgroup.com/en/expertise/vibrant-thoughts/world-cancer-day-2021.html
[8] https://www.rbccm.com/en/gib/healthcare/episode/the_healthcare_data_explosion
[9] https://www.forbes.com/sites/gilpress/2016/03/23/data-preparation-most-time-consuming-least-enjoyable-data-science-task-survey-says/
[10] https://www.inzata.com/is-ai-changing-the-80-20-rule-of-data-science/
[11] https://www.Syntropy™.com/news/Syntropy™-md-anderson-news-release-04-29-2021/
[12] https://www.Syntropy™.com/news/Syntropy™-uci-news-release-10-06-2021/
[13] https://en.wikipedia.org/wiki/2020%E2%80%93present_global_chip_shortage
[14] https://www.emdgroup.com/en/news/digital-ethics-advisory-panel-08-01-2021.html
[15] https://www.emdgroup.com/en/sustainability/business-ethics/advisors-for-bio-and-digital-ethical-issues.html}