Data Mapping and Metadata – Purpose, Value and Opportunities

Published On November 27, 2020

Whether your organization is challenged with digital transformation efforts, struggles with substantial data volumes, or navigating technological change and a range of users, data mapping, data flow mapping and metadata management all provide tremendous value. This is true for the Information Governance (IG) and Records & Information Management (RIM) community that support these initiatives as well. These capabilities apply to any organization and business or technical process that needs to know where data resides, who owns it, within which applications and with what context. However, there are challenges when working with stakeholders. For instance, lines of business like IT or Legal and Compliance frequently have varying perspectives and understanding when discussing data mapping. This blog provides a view on the value of data mapping and metadata in part one and an illustration of opportunities and tools in part two.

From an IG perspective, data mapping and metadata management are about describing information with an objective of managing the information lifecycle and complying with various business enablement, governance, control and risk mitigation needs in the organization. What are these capabilities from the IG/RIM perspective? Opportunities and challenges regarding these capabilities, along with practical applications and real life examples will follow in part two of this blog. An important takeaway is that IG/RIM needs to be assertive with IT, lines of business and support functions to deliver value – including that our focus, information lifecycle management, is critical to digital transformation.

What is data mapping?

In computing and data management, data mapping is the process of creating data element (field) mappings between two distinct data models, tables or databases. This allows architects, engineers, developers and others to understand exactly where specific data elements come from and go to whether for development, replacement or querying applications and data.

Closely related, but truly different, is data flow mapping – illustrating the transition of data among data sources, not at the data element level. When mapping data flows, the interaction points between all objects (applications, servers, users, etc.) are to be identified via the flow or passing of data. By mapping the flow of data, you identify any unforeseen or unintended uses. A data flow map also helps you to consider the parties that will be using the information and the potential future uses of any data processed.

Metadata management involves managing descriptions or descriptive data about other data – in the IG world we would refer to data as content. The term metadata is most often used in relation to digital media given the world we live in today, but other common forms of metadata are catalogs, dictionaries and taxonomies that contain descriptive data. Metadata can also be used to control the lifecycle, access and use of data, such as a retention period, privilege level or digital right to the data. Perspectives on metadata include the following:

• Operations personnel would think of inventory management – boxes, files, media, equipment

• IT and business management think of application inventory and data source descriptions

• Chief Data Officers thinks of describing data elements, essentially another level of inventory with similar and expanded controls for governance and use or elements

Addressing data mapping challenges

Many of us in IG/RIM, as well as those in the IT community, have had confusing conversations around solving problems with data mapping because it seems to have many interpretations. Lacking a common understanding will lead to challenges in our varied initiatives. To start, we need to identify the problem to resolve clearly. In addition, we need to focus on incorporating the broader results expected from our plans when considering automated approaches using available tools to address objectives including:

• privacy protection

• identifying and maintaining Systems of Record (SOR)

• enabling retention, disposition, eDiscovery

• application decommissioning

• sharing retention obligations and other compliance requirements

• inventory management

• minimizing dark data

• content classification

• legal holds – both unstructured and structured data

• enabling and partnering with Data Governance or Cybersecurity

Another important value consideration that is technically oriented is the accurate transition and linking of data elements and data sets for use and integrity by other applications and processes. These days, regulators are paying more and more attention to governing processes and technology in order to reduce risks and comply with a myriad of dynamic obligations, some on a conflicting and/or global basis. This includes levying substantial fines for lacking governance and control. IT leverages application databases that contain extensive metadata in order to address the myriad controls and demands on applications and data. They also use these application databases for maintaining and measuring the development of applications. Effectively considering these points will address business challenges. These challenges include the proliferation of dark data – the unknown, lost, misidentified and misused data. As before, our objectives are to understand what data we have, where it is and the relationships among sources and uses.

Conclusion

In summary, data mapping, data flow mapping and metadata management are frequently misunderstood capabilities that need to be made clear with stakeholders in any engagement. Understanding what these capabilities do, how to apply them and the value are as critical as identifying the objective or the problem to solve in an organization. Following part one, the next blog in this mini-series will discuss practical application and supporting tools.