Oct 14 — 2020

Data Science versus Business Intelligence

Intro

Data is everything, everywhere. Imagine that you leave your house in the morning and you start pondering which route you should take to work: Ok, that simple decision based on a quick estimation of time and distance is data-related. From a simple commute, we can see that decisions, either driven or influenced by data, are made unconsciously in our day to day.

Over the past 4 years, I’ve taken part in data strategic initiatives for leading international firms across various industries. Nowadays, I look for opportunities in businesses, assessing how data can enable our clients to do better. It is common knowledge that in the second half of 2020 companies are still not seizing the strategic potential of data and, according to MIT market research, less than 5% of companies use it well enough for gaining a competitive edge. Moreover, I have also experienced how companies manage these projects and the common key challenge relates to stakeholders holding diverse points of view for some basic concepts . Everybody in today’s organisations heard or talked about Business Intelligence (BI) and Data Science while no one has properly given a thought to their meaning. In these circumstances, interactions during the commercial, or operational phase tend to cause frustration, misalignment of expectations and failure. A mistake that organizations clearly make is underinvesting in organisational culture and mindset change which is fundamental for individuals not overlooking the potential of adopting data-driven mechanisms at every layer in the organisation.

Therefore, the main root cause for this lack of success seems to be misunderstanding of buzzwords . After coming to this conclusion, I felt that it could be educational going back to the fundamentals of BI and Data Science, shedding some light on how they compare to each other.

What is commonly understood as Business Intelligence?

Business Intelligence (BI) is a generic term that dates back to 1989, whose original definition was along the lines of “mechanisms and the underlying technology to improve business decisions”. Nowadays it is understood as the development of dashboards, digital reports or ad-hoc analytical visualizations. This may be basic KPIs digitally displayed while other companies may use advanced analytical methods based on statistical models, either way with some governance and security around insights delivery. Regardless of methods or technology, BI aims to provide bullet-proof facts for informed tactical/strategic decision making and the priority is providing actionable information in the hands of the management calling the shots to quickly act on patterns or insights. Concisely put, BI aims to explain past events using data that emanates from the business regardless of it coming from marketing, sales or operations.

Ok, so what is then Data Science about?

Firstly, it is considered a science because it aims to discover the unknown using methodical research and analysis techniques. Its recent success in society is derived from humans being intelectually curious and unwilling to ignore the unknown. It is about explaining and predicting events using a combination of mathematics, statistics, computer science and business knowledge within a domain-specific context.

The associated job title ( Data Scientist) has been around since 2008, when it was introduced by D.J. Patil, and Jeff Hammerbacher at LinkedIn and Facebook respectively. It started to be a fancy trend when in 2015 The White House announced the first Chief Data Scientist and now demand for talent outpaces supply. Lately, Harvard Business Review even claimed that it is one of “the sexiest jobs of the 21st century” and some articles claim that data scientists are the new investment bankers but it requires a vast set of skills that is hard to combine. Some of these bright individuals are PhDs in “exotic” fields like biomedicine or astronomy, but the majority have an academic background in computer science, maths or physics. The data science role is about shaping large quantities of messy and disparate data sets to make their analysis possible, developing modeling and prediction mechanisms,concluding what happened and what is likely to happen next.

Data Science vs Business Intelligence: commonalities and differences.

In common: Both practices provide fact-based insights for motivating, easing and supporting business decisions but these practices tend to focus on different temporalities. Also, both approaches require a visualization layer, data management and governance.

Differences: In BI, it comes down to a validated formula or a known method of calculating a KPI. BI provides a reporting mechanism for showing updated values of previously known metrics, dealing with predictable “known unknowns”. BI assists with descriptive analytics mostly.

On the contrary, in Data Science, the business comes with questions that were never asked nor answered before so Data Science deals with unpredicted “unknown unknowns”. Data Science, as a field of automated statistics in the form of models, goes further to predictive and even prescriptive analytics, enabling future prediction and aiding in classifying and predicting outcomes.

In essence: BI is about interpreting and visualizing data whereas data science is about using statistics and other analytical tools to forecast what is likely to occur next.

Conclusions

Data Science is not a newer form of BI. Both are critical milestones for any organisation that aspires to be data-driven so:

Business Intelligence maturity fits well within a Data Science roadmap as a preliminary step to predictive analytics. If you really think about it, firstly, you must understand and analyse past data and extract insights, to later build models that allow you to predict the future of your business.
The typical BI project framework is not applicable on Data Science projects because the latter demand specific operational requirements right and the typical IT, Software as a Service, plug-and-play, +configuration, mentality must go through the window.

In Bedrock we strive to pave the way for the democratization of Data Science and AI ( in the form of Machine Learning) and our approach is built upon a close collaboration between our team and our clients’ business domain experts, counting on the appropriate support from management. This approach is what guarantees that our projects end up delivering tangible results.

Bonus: Many Data Analysts and BI specialists teams have rebranded themselves as Data Scientists and this leads to confusion. Traditional Data Analytics differs from Data Science. Yes, some tasks overlap almost half of the time such as data sets wrangling, crunching, exploratory analysis and data visualization. However, the difference resides in coding, modeling and in using algorithms which is why Data Scientists mostly use Python or R, developing models for correlation, causation, and counterfactuals, trying to guess what is gonna happen. Data Analysts are not data scientists and being able to use SQL, PowerBI or Tableau is only a tiny fraction of what is required.