Big Data Tool Overkill When You Just Need Embedded BI

Big data won't be the right tool when you just need ad hoc reporting.New data management technologies open up a wide array of capabilities for businesses. But don’t be distracted by shiny objects!  Selecting the right toolset for any challenge requires starting with a clear understanding of the business problem that needs to be addressed.

In recent years, we have seen major disruptions in how businesses manage their data. Big data technologies have emerged to address new challenges arising from the rapid growth of datasets and the sudden influx of new data types and structures that these organizations must deal with. Such technologies also address the need to find answers in real time from such datasets.  New data infrastructure technologies such as Hadoop, Spark, Mongo, Kafka, and others address these challenges and are opening up new ways for businesses to manage increasingly large and complex datasets with speed and efficiency.

This rapidly changing landscape is creating an unprecedented opportunity for independent software vendors to take a step forward and embrace new technologies. But while those technologies have tremendous promise and deliver value in a variety of ways, that doesn’t mean that they are always the best choice. Sometimes ISVs need to incorporate massive volumes of multi-structured data for advanced analysis. But sometimes what they really need is embedded self-service analytics, including ad-hoc reporting, dashboards and other visualizations.

The Old Sidestep

The thing about taking a step forward is that it needs to be an actual step forward. If what you need is self-service reporting and analysis, implementing technologies optimized for the big-data challenges listed above can be more like taking a step to the side…or walking off the trail altogether. No one puts it better than Forrester’s Boris Evelson when he cautions us not to throw Hadoop at every BI challenge. He explains that many Forrester clients start out convinced that they need a “big data” solution. And then something interesting happens:

…[A]fter a few probing questions, companies realize that they may need to upgrade their outdated BI platform, switch to a different database architecture, add extra nodes to their data warehouse (DW) servers, improve their data quality and data governance processes, or other commonsense solutions to their challenges, where new big data technologies may be one of the options, but not the only one, and sometimes not the best.

Blogger Sachin at Fromdev takes that idea one step further. In outlining the best use cases for Hadoop, he also provides a quick list of times when it is not appropriate to use Hadoop. The list includes:

  • When current systems are adequate to handle the overall volume of data
  • When those systems can manage the speed of data growth
  • When what you really need is to support simple data reporting and analysis use cases

It would be a mistake to think that those three scenarios apply only to Hadoop, especially that last one. If you try to do operational reporting from Hadoop or Mongo — or any of the platforms mentioned above — you are introducing significant complexity both to your end users and your infrastructure as a whole. Data integration, data modeling, security, lineage, and other core requirements often call for workarounds or special treatment within a big data environment. And the question is: what is all that complexity getting you? What advantage is the big data platform bringing you for that use case?

The Right Environment

When choosing a technology solution, the fundamental question should be, “Does it solve the underlying business problem?” If your business is an ISV, you face a broad range of problems that it needs to address both for themselves and their customers. These include various challenges related to embedded analytics and reporting. Some users need real-time, self-service reports. Others need to create dashboards within web-based applications to provide reporting against a transactional system. Still others need to implement full-blown self-service data discovery. You don’t want to be bogged down in administering the wrong infrastructure; you want to focus on enhancing your application.

Whatever the specific use case, if you’re trying to empower a broad range of employees or users to make data-driven decisions, the established technology for making this happen is a relational database management system (RDBMS) integrated with an embedded BI solution. Unless you have specific big data requirements that such an architecture won’t address, chances are that model will be the best fit. All of the challenges mentioned above around integration, data modeling, lineage, security, etc. have been long addressed for this model and are essentially solved problems.

In addition to that core interoperability, you need a solution whose business intelligence and analytics blend seamlessly into your application’s interface, and assure stronger user adoption and satisfaction. End users must be free to create and explore reports, dashboards and visualizations in real-time, ideally without needing any IT or database experience and without having to learn a new interface.

The Right Tool

At Izenda, our integration-ready web application enables organizations to deliver modern, self-service reports, dashboards and visualizations to applications whether they are on-premise, in the cloud, or offered via a SaaS (software as a service) model. End-users securely interact with customized analytics through a rich, 100% web-based interface, without needing IT or database experience. Many ISVs (and others) — having tried a big-data approach to solve basic operational reporting and other core analytics use cases — are turning to Izenda to provide a fast, reliable, and fully integrated solution for putting operational reporting and other core analytics capabilities directly into their customers’ apps and right into the hands of the users.