Tips for Managing Unstructured Data

Stack of papers with different colored paperclips

In this era of rapid information access and technology sophistication, accessibility to quality data is essential. Some asset managers have spent the time and money necessary to develop architecture and data models to satisfy needs related to structured data. Many more have yet to tackle the more challenging task of trying to manage their unstructured data. If you want to stay ahead of your competitors and already have a strong data governance model surrounding structured data, your firm is probably considering how to turn attention to these more complicated data sets.

Let’s start with defining structured data and use this to explain why unstructured data is different and important. Structured data is any information that follows a common format and can be stored in a tabular database—think performance metrics, zip codes, settlement dates etc. Unstructured data is simply any data that is not conformed into a specific or prescribed format and model. Examples of unstructured data are analyst reports, research papers, news feeds, email text, and audio or video clips. Any piece of information your firm receives that is not formatted and organized to be database consumable is unstructured data. Think about how much unstructured information your firm utilizes and how much can be reasonably consumed versus what’s available. Imagine if you could synthesize that data and analyze two, ten, or even a thousand times the information in the same amount of time.  That’s what the power of managing unstructured data can do for you or your firm.

If you’re ready to unlock this potential but aren’t sure where to start, I have the following suggestions to get you thinking.

1. Prioritization

When considering all the various forms of unstructured data—email text, images, documents, videos—each of these may require different approaches and technologies to manage. However, not all of them will yield the same amount of value for your organization. For instance, if you’re an asset owner, you’re not likely to get as much value from analyzing email content or call center audio files as a retail asset manager. Your time and money may be better spent on a strategy for consuming reports and news feeds. Determining and consistently evaluating what information is important to your business is a foundational step to a successful model for managing data.

2. Storage and indexing

Organizing and cataloging unstructured files is crucial for being able to retrieve what you want quickly. It’s not good enough to create shared network drives with folder structures that suggest where a document should be saved and made available. We all know from experience this doesn’t work—files get deleted, renamed, or saved in the wrong place. There are plenty of robust cloud-based content management solutions that can catalog, index, and store unstructured files in an automated way, making them secure yet accessible and searchable. This is an easy solution to implement and should be considered as a minimum solution for trying to tackle important unstructured data.

3. Transformation

The ultimate way to take advantage of unstructured data is to transform it into structured data. This effort should be focused on your largest and most valuable data sets that can produce positive ROI. There are emerging technologies that use artificial intelligence and/or machine learning capabilities and smart algorithms to consume unstructured data, read and categorize its content, then extract and publish the findings in a structured way. This can enable you to identify like data points across disparate formats and sources and conform it into structured data that can then be used to model trends or outliers. This is the future of data analysis.

As we move forward in an increasingly competitive market, I’m reminded of the age-old adage that “knowledge is power.” This is as true now as it has ever been. The modern-day challenge is figuring out how to wade through the growing mass of information available in a quick and efficient manner to ensure it’s actionable before it’s out of date. The only way to do this is by developing a forward-looking strategy that optimizes both structured and unstructured data.