Self service BI in an agile world

Analysts Claudia Imhoff and Colin White defined self-service BI as “the facilities within the BI environment that enable BI users to become more self-reliant and less dependent on the IT organisation”. It's not a new concept. Ever since Microsoft Excel became ubiquitous on the desktop of any self respecting accountant users have been autonomously manipulating and analysing data to answer business questions. But Excel can only take us so far. Even without the row and column limits of the earlier versions of Excel, the rapidly increasing volume of data generated by even small companies today make Excel impractical. Traditionally, in these circumstances users would fall back to reports and dashboard created by the IT department. But with the increasing pace of business change, this approach is too slow, too prescriptive and stifles the business' ability to innovate or react swiftly to the unexpected. Little wonder then that the last few years have seen a rapid rise in the fortunes of companies offering self service business intelligence, and more traditional BI vendors increasingly struggle to maintain market share.

Necessary as it is, self service business intelligence presents some formidable challenges. Primarily, business applications hold information in structures that are efficient to store and retrieve, rather than structures that are designed to convey meaning. Modern organisations also rarely use a single business application, so data needs to be collated across multiple data sources with different connectivity challenges. Translating these disparate structures into a single combined meaningful information source is a task that requires a deep understanding of the original data design and the applications that populates them. This knowledge is typically held by a select few individuals in the organisation, if at all. There is also the issue of governance. In an environment where business users are helping themselves to data and autonomously translating it into meaningful information in the format they choose, ensuring data integrity and security becomes a real challenge.

All of these problems are typically overcome with a data warehouse that collates data from a range of business applications and stores them in structures designed to convey meaning. The transformation of raw data into meaningful information and challenges of collating data across multiple data sources is largely handled by routines that populate the data warehouse, designed and implemented by IT professionals. Business users can connect their chosen analysis tools to a single data source that contains meaningful information, consistent across the organisation, and data security can be provided by controlling the contents of the data warehouse and who has access.

However, as we've discussed previously in this blog, a data warehouse presents its own challenges. They only provide access to a subset of the information available in the business applications and require batch updates which suppress the BI platforms ability to respond rapidly to change. The ideal is a platform that allows the knowledge of the IT department to be used to build a live analytics framework. Rather than the routines to build the meaningful information for users always running prescriptively at a set point in time they can, where appropriate, be run on demand to provide live information. Such a solution can deliver true self-service business intelligence whilst maintaining the agility of the BI platform.