Data Integration – A Necessity or a Smokescreen? Part 3
As we have already covered, InforData Consulting believes that any information management strategy team needs to realise that Data Integration is an integral and vital part. In this article we attempt to quantify why and highlight some cost effective alternatives to the traditional approach.
The basics of Data Integration are very simple, disparate data sources need to be bought together to form a single repository; but is that really all there is to it? And how does this single repository benefit an organisation.
It is common knowledge that the amount of data that organisations are collecting and utilising is expanding rapidly, much faster than it has ever done before. Therefore the need for consolidation/integration is also growing. Data sources are becoming more and more disparate, greater in number, harder to manage and potentially most importantly much more complex and difficult to maintain in terms of data quality. Without integrity, data is almost useless. Users don’t trust the results and management teams make decisions on inaccurate data, all potentially expensive costs to the business. As the data grows, so does the risk. Some analysts advocate spending money on sophisticated tools to help combat this, but as ever InforData Consulting remains unconvinced that purchasing a tool will resolve inaccuracies and deliver the single repository required. Of course tools will assist the process but essentially it is a combination of people and process that will ensure the successful delivery of an integrated data set.
Although the upfront cost of implementing an information management strategy can seem high, when balanced against the cost of poor quality data, inaccurate information and potentially uninformed decision-making, the business case almost writes itself. It is estimated that over a 5-year period a business will save in the region of 5 times the amount it spends on this type of project. A decent return on investment in anyone’s book.
So is Data Integration as simple as it sounds? Essentially yes. In fact, the whole, sometimes seemingly mysterious, world of Data/Information Management is actually very simple. No matter how many data sources need to be integrated, no matter the type of data, the process remains the same. Create a single repository, create a set of common business names, rules, and move data from one or more silo’s to a central repository.
Of course, the devil is in the detail and no data integration project is ever going to be simple or straightforward. The sheer volume of data and amount of sources will ensure that the level of complexity is greater than the fundamental “rules”. However, in order to get your business users to understand the concept and buy into the funding of the project, simplicity remains the key, coupled of course with the benefits.
Data Integration projects can take a significant length of time to implement and the underlying strategy incorporating Data Quality and Data Governance must be robust enough to carry the organisation through a lengthy period of change and afterwards continuity, yet do not forget that the business world moves rapidly and whilst robustness is a key factor to ensure data quality remains high, it should also be realised that these strategies and processes need to be iterative, highly adaptable and open to change. This can be easily monitored by the BICC (see our previous article) and changes proposed and ratified dependent upon urgency, requirements and necessity.
A truly integrated, iterative strategy will allow the business to rapidly reap the benefits mentioned above. Ongoing maintenance of the repository can be arduous and work heavy, but nonetheless remains a necessary task. One can consider streamlining the processes in the underlying systems to make, for example, adding a new product line simpler or alternatively manage it at a higher level in the repository. More complicated is the integration of further systems or silo’s of data (i.e. in the case of a company merger). This is a further data source that needs to be integrated into the single repository. However, it is a time consuming process and unless an alternative way of integrating the new data is forthcoming, decision-making based on an accurate and single repository becomes trickier. Therefore, thought needs to be given to how the additions/changes affect the output from the repository. It can be better to create a further data source, for example a spreadsheet (particularly for forecasting or budgeting requirements) utilise the common business names as depicted in the strategy and merely upload into the repository, completing the true interface and integration further down the line, whilst of course adhering to the guidelines in the strategy.
There is an alternative…. Investigate Business Intelligence and Performance Management tools. There are options that will allow you to connect to multiple or unlimited data sources, create an integrated single repository, put in place data quality checks and allow you to make changes to the underlying data (i.e. the new product line mentioned above) that will filter through the entire stack (underlying source, repository and all output such as reports, dashboards, budgets) at almost a click of a button. Granted there are few options that will deliver all of this in one module, but if you look hard enough you may just find something that suits you, an all-in-one approach that can alleviate some of the headaches associated with a full data integration programme. In some cases it is a sensible route to take to identify gaps in the data, data quality issues or indeed to understand if the integration process is producing the single version of truth required by the business.
Some food for thought perhaps, and hopefully a combination of traditional approach, quantifiable reasons for embarking on such a project and an alternative to the tried and tested methods.
InforData Consulting can assist with advice on any of the subjects covered in this article. Our expertise crosses the full range of Data Management including Business Intelligence and Performance Management. A personal touch with a different outlook.