Data Science & Big Data Analytics
What Are Structures in Data?
Structures in data represent patterns and relationships that can provide insights into causal connections and inform decision-making. In healthcare, this might mean identifying data points that signal the presence of a disease. In FinTech, patterns may justify a decision to buy or sell a stock. Regardless of the field, data can reveal mechanisms that underpin processes, which can ultimately be visualized and explained.
Self-Organization of Data – Making Data Structures Understandable
By mathematically arranging high-dimensional data on a plane based on their similarities, patterns can emerge. We refer to this process as data self-organization, and the successful formation of these patterns is known as an emergent phenomenon.
An example from healthcare demonstrates how similarities in patients' gene expression profiles can be visualized. Each profile includes 7,500 measurements (dimensions), represented as a 3D-printable landscape map. This map enables an intuitive representation of relationships and provides critical insights.
How the 3D Landscape Works:
- Self-organization through swarm intelligence (see scientific details): Structures in data are identified without relying on diagnoses.
- Topographic map of patients:
- Data points represent individual patients.
- Similar profiles are located in valleys.
- Mountain ranges separate dissimilar profiles.
- High-dimensional discontinuities become visible as hills and valleys.
- Leukemia diagnosis types and healthy patients are marked as differently colored points for clarity.
Advantages of Visualization:
- Data structures reflect tumor identities, making them easier to understand.
- Two outliers suggest potential issues in diagnosis.
- The conclusion is that, aside from the outliers, the diagnoses are well-represented in the data and can be learned from it.
Supporting Human-in-the-Loop Decision-Making:
This innovative visualization makes complex data structures intuitively understandable and supports a "human-in-the-loop" approach (see scientific details), where humans and AI work together. This facilitates well-informed decisions in healthcare and enhances diagnostic accuracy.
This example represents just one of the cutting-edge Big Data analyses we offer.