An Introduction to Data Fabric
/
Managing, integrating, and leveraging information from disparate sources is a common challenge in today's data-centric world. Digital transformation is happening everywhere.
To address these challenges and forces of change, organizations will benefit from implementing data management strategies to create secure environments where integration, storage, processing, and accessibility of sensitive data can seamlessly unify an enterprise through controlled data access. In short, implementing data fabric helps companies unlock the potential of their data assets to provide actionable data-driven insights.
What is Data Fabric?
Data fabric is a holistic, integrated approach to data management. Data fabric is a set of foundational components that provide predictable, reliable, and authoritative capabilities through data services across multiple endpoints spanning on-premise and cloud environments. Data fabric is:
Predictable: the data formats and web services are not changing.
Reliable: there are consistent refresh rates with no downtime.
Authoritative: the information comes directly from the source, which is vetted as the single source of truth.
Data fabric is not additional pieces of software or data warehouses. Think of it as a virtual layer that connects all the data securely, providing a consistent, comprehensive view. It’s an adaptable, scalable framework for your technology to work together.
Data fabric helps organizations struggling to integrate their big data and has the ability to produce actionable reporting. Many institutions get stuck on one or more multiple foundational components. Yet, one of the greatest challenges comes with taking the next step to predict outcomes, optimize actions, and automate systems.
Evaluating an Organization’s Data Maturity
Performing a maturity assessment helps evaluate systems and data to identify strengths and weaknesses. It can also serve as a guide to integration.
Before implementing a data fabric, it is essential to perform a maturity assessment on your systems and data. A review will help you evaluate the quality, reliability, and effectiveness of your data and data-related policies and processes. This is not an intimidating process or steeped in detailed data analytics. You assess how well your organization manages, analyzes, and utilizes data and how that data drives strategic decision-making. Some assessment components include:
Data governance.
Data quality.
Data integration.
Data analytics.
Data management.
Data storage.
Data collection.
Data validation.
Data reporting.
An assessment allows you to better understand your data’s current state and serves as a strategic guide for continuous improvement. An example of one of the maturity tools to consider is Slimgim, an open-source method consisting of approximately 50 questions.
Looking back more than five years is important when performing a maturity assessment. Having a historical view can help you better understand your long-term progress.
Data Fabric Architecture
Data fabric provides a foundation to support the entire data lifecycle.
Organizations face the challenge of fully harnessing their data's power to provide actionable insights, support business decisions, and increase the value of their business intelligence. As a unified framework, the data fabric is critical to a data management strategy. Data fabric is comprised of five pillars:
Services/application programming interfaces (APIs).
Data standards.
Software.
Security.
Culture.
These pillars support an enterprisewide data lifecycle.
Services/APIs
Software development companies are increasingly transitioning to cloud-based systems. The overall trend is moving towards a service-central and software-neutral approach. From this perspective, it is essential to implement APIs, which allow different software applications and data sources to work together, share information, create integrated solutions, and is fundamental to digital growth. In simple terms, APIs:
Streamline data and system integration and collaboration processes.
Standardize how systems interact, exchange data, and leverage each system’s capabilities.
Combine functionalities from multiple sources.
Facilitate third-party developers in creating additional applications.
Data Standards
Data standards are key to integration because they decrease the effort to transform and validate data. Data standards, part of data governance, are an organization's rules and guidelines to establish how its data is structured, formatted, and represented. Acting as a common language, data standards enable enterprisewide use of data consistently and predictably, as well as creating data storage best practices and authoritative data models. Establishing them eliminates the back-and-forth when integrating disparate data sources.
For example, consider O’Brien County in Iowa. One system may have it as O’Brien, whereas another system may store it as OBrien. If the County name attribute had data standards, each system would label it using the same characters, increasing data processing efficiency.
Software
Staff need access to appropriate software, including an appropriate number of licensing and availability to easily add software to an organization’s software portfolio. Not having enough software licenses causes unnecessary delays while staff members wait for a license to free up. Leverage licensing agreements that allow you to scale licensing as needed.
Also, use the configure first software strategy. This is when you buy a software framework and configurable tools that can build customizations when needed. For example, in the Esri® ecosystem, you can build configurable maps and apps; however, sometimes your needs are more specific, i.e., extending templates and widgets and customizing with APIs. The more you extend and customize, the more effort and cost you invest.
Security
Think of enterprise security as an asset. It’s there to protect your infrastructure, data, and other assets. An example of enterprise security is single-sign-on (SSO). From a user perspective, it’s highly frustrating to remember different usernames and passwords for each platform. One of the SSO benefits is increased productivity when users can move freely from software or platform to another software or platform.
Another security component to consider is open data. Many organizations often secure more of their data than necessary. The more you restrict your data, the more resources you use to administer all gateways. Another benefit of keeping open data is that everyone in the company can use it, increasing accuracy and forecasting abilities.
Culture
Culture plays a significant role in how organizations handle and leverage their data. The companies that are the most successful with data management are where the culture begins at the top level. Their actions, behavior, and how they set expectations have a trickledown effect, allowing everyone to keep data at the forefront of everything they do. Another way they encourage a data-centric culture is through policies and procedures.
Some reasons why culture is important for successfully implementing data fabric are:
Ethics.
Decision making.
Trust.
Collaboration and sharing.
Continuous learning and improvement.
Data Lifecycle
A data lifecycle represents the processes involved with data management. With the data fabric pillars in place, an organization is ready for effective integration with the data lifecycle, which includes:
Data Collection: cloud, sensors, open data, remote sensing, and web service central.
Existing Systems: business processes such as financial, reporting, and security.
Visualization and Reporting: augmented reality, mixed reality, modeling, mapping, and dashboards.
Master Data Management: machine learning/artificial intelligence, LRS, database, and spatial information.
Each of the pillars allows data to be effectively collected, stored, accessed, processed, and analyzed. Working together, the pillars allow authoritative data integration throughout an organization.
Leveraging Integrated Data Management Systems for Success
The amount of data will continue to grow at an exponential level and be a transformative force. It is critical that organizations implement an integrated data management strategy to be successful. By embracing data fabric, we eliminate silos, bring together disparate data sources using data science, and unify data environments. With data fabric as a foundation, companies can continue the digital transformation and harness their data’s full power, converting it into a strategic asset.
For more information, check out our on-demand INSIGHTS webcast on Data Fabric.
About the Author
Eric Abrams
Senior Project Manager
Email Eric
Connect on LinkedIn
Get to Know Eric
