Data Integration – A necessity or a smokescreen?
Over the last few weeks, InforData’s resident writer has tackled the tricky concept of getting the business to buy into a Data Management Strategy. Now they move on to Data Integration…. An equally challenging subject, that most definitely has its roots in IT. But is Data Integration a vital part of a Data Management Strategy, and does it have to be the painful, costly exercise it is so often perceived to be?
It is generally accepted that Data is one of the single most valuable assets of any organisation, but why? For data to be valuable and useful, to support decision making processes, aid cost reduction, provide competitive analysis and generally underpin process and procedures fundamental to any company, it must be accurate, controlled and potentially integrated.
Throughout the business world, data is growing. Recently, the concept of “big data” has been introduced. Monitoring traffic to websites, social media, unstructured data, minute by minute transactional records, all of these things and much much more are being measured, analysed and reported on. Forecasts and budgets, instrumental to the success of any organisation are devised on this “data”.
Integrating data from disparate sources, 10’s or even 100’s of applications, spreadsheets, databases, legacy systems and even good old fashioned pen and paper, can be a daunting task. Where does one start? And unless you integrate ALL the data in the organisation is there any point in starting in the first place?
Data Warehouse and/or integration of applications into a single repository, with a data governance and quality strategy wrapped around, will give you a “single version of the truth” and will help future proof the process for coming years. However, this is a challenging task, especially when there are a multitude of sources and a plethora of business requirements to take into consideration. Taking a section at a time is a popular approach too, but is unlikely to give you a true single repository. In effect a piecemeal approach will still result in silos of data, albeit that these silos are fewer, more controlled and ultimately more accurate. The final stage of this approach allows consolidation of these new silos and eventually the ultimate goal of a single repository can be reached.
Alternatively, one can invest in Business Intelligence tools to “integrate” the data. Taking data from disparate sources and allowing you to consolidate in a BI solution can be useful. Taking this approach can highlight issues with Data Quality in particular. But there again most BI solutions are made up of disparate applications themselves, one module for reporting, another for ETL, another for visualisation, potentially exacerbating the challenge of uncontrolled data. There are tools available that deliver the range of BI (including a single repository, ETL, visualisation) “all in one” without the need for separate modules, but these are few and far between.
So, what should you do? InforData Consulting believes that this is a much simpler question than it appears. As with any project, start at the beginning; assess the needs of the business, identify the data sources, how to consolidate tabular data by utilising business names etc, in other words KEEP IT SIMPLE. An unusual stance for a consulting company, but we pride ourselves on taking a different, individual approach, assessing what is required and basing recommendations on a combination of these requirements and the company ethos.
As ever the key to a successful project is understanding the elements involved and tapping into internal and external expertise where required. A robust strategy and a clear goal are integral to achieving the best end result.