Unstructured Structured Data for Data-Driven Organizations

June 28, 2016October 14, 2018 Scott Brandt big data, Blog, In the Flow, metadata, structured data, unstructured data

We hear so much about “big data” and unstructured data these days, and we automatically associate things like emails, texts, tweets, and audio clips to the unstructured data category. And as we are being reminded by the purveyors of the big data mining tools, it’s all about making sense of this huge mess of uncoordinated, anomalous data.

Though much of this relates to sales and marketing data, often the information associated with workflows is more structured, meaning we can easily get quantifiable measures of activity and performance. There are numerous dashboard providers that take this data and deliver a myriad of visual representations. These pictures help managers and administrators oversee and identify points of improvement.

Unstructured Structured Data

Before we get to true data mining of all the other “big” data, we should be looking to mine and benefit from all the “unstructured” information within the files we currently use in our everyday processes and workflows. I call this unstructured structured data because the information is found in nice, handy files and more often than not in standard formats and organization. The data also epitomizes the utilization and necessity of these data repositories (i.e., the files that represent documents, drawings, spreadsheets, flowcharts, etc.)

For example, we may associate a specific document to a workflow every time we execute the workflow. But it is not the document (file) itself that may be important but the data within the document (the metadata if you would). So what is really important is not that the document is identified and assigned to the process, it is the data in the document. Yet, we often leave it to the user to manually search and find the data they need within the document rather than provide it to him based on heuristic analysis of what is needed and what has been utilized before. This is operational “big data” in the sense that knowing what individuals need before they look for it significantly increases efficiency, and more importantly, accuracy.

Data-Driven Organizations

As data becomes easier to acquire, distribute, and analyze, we see more organizations with formal decision-making processes based purely on data metrics and statistics. For example, if customers purchase more of item X in the afternoon than in the morning this suggests a rearrangement of the product and replenishment cycles. But in our rush to find the tiniest bit of insight from our data we see organizations overlooking some of the best data mining opportunities available to them; that is, focusing on the information they currently have and use in their daily operational workflows. Finding and analyzing the data within our existing files, documents, specifications, and manuals can provide great insights and improvement opportunities. The data is easily gained through full-text searches, through heuristic searches, and workflow reviews. Knowing which documents, and more importantly which specific data in those documents, are most used, are the most useful, and are most revised lets organizations improve and optimize routine, but important, processes.

Business Process Improvement Using Data

So although we are doing a better job of monitoring, managing, and reporting on key workflows within the organization, essentially by using standard metrics, we also have the opportunity to improve these business processes by examining and proactively providing needed data within those processes. Making sure steps get accomplished on-time and generating appropriate notifications throughout the workflow certainly improves our performance, accuracy, and amount of rework, but identifying critical information and data required during the steps and inserting them in the process – before the user even knows he/she needs it – automatically improves the speed and accuracy of the workflow.

So the next time you provide that 100+ page specification document to a user who needs it, look at it as a data mining opportunity, where if appropriately identified can lead to an analysis that not only enhances workflow but reduces employee frustration and search times, not to speak of eliminating unnecessary printing.

Scott Brandt is president and CEO of eQuorum.