Your metrics are lying to you.
The overwhelming majority of companies simply don't have the tools or people to make data-driven decisions. For example, a small company will have hundreds of thousands of rows of CRM data for time-series analysis. Salesforce doesn't support analyzing this data, and most companies can't perform the analysis themselves. This creates a false sense of understanding.
To better comprehend this illusion, and how dangerous it is, we need to distinguish between a few types of analysis.
Point in Time Analysis
An incredibly common analysis is a Point in Time (a snapshot) analysis because it simply represents a view of the organization as of the time the report was extracted from the system.
Most of the organizations we work with are focused on growth, and Salesforce is the most common CRM of large organizations, so a common report we see is a funnel, bar, chart or similar. This is a perfect example of a Point in Time. Generally, it looks something like this:
Most executive decision making based on "gut" instincts is really from decades of viewing these types of snapshot dashboards and drawing conclusions from the recognition of patterns. Essentially muscle memory connecting the dots.
Source to End Analysis
The second type of analysis is a Source to End analysis, which explains the difference between two numbers or reports.
We see this all the time in working with FP&A teams in the form of "Variance" or "Budget vs. Actuals" analysis. At the individual contributor level, this often works because they are comparing month-to-month changes. However, at the executive and Board of Directors level, we commonly see a waterfall chart over the course of a quarter or year, which is used to explain what they saw last time and what they are being shown today.
Plainly speaking, Source to End analysis creates a simple explanation between where you were before and where you are today. For most organizations this is considered an advanced analysis, and executives are relying heavily on these reports to make crucial decisions.
Following the Salesforce CRM open pipeline example, a common bar chart would look something like below.
The executive waterfall chart explaining the difference would be created to "bridge" executives between the two bars. We'll dive deeper into the waterfall chart in the next section.
Evolutionary Analysis
The third type of analysis is an Evolutionary Analysis, which is much more accurate, albeit incredibly difficult to perform. An Evolutionary analysis explains exactly how the data has evolved in between the two comparison periods. This provides a precise explanation of the difference between the two numbers or reports.
Anybody who’s ever taken a wrong turn, missed an exit on the freeway, or missed their connecting flight knows: How you get from your start to destination really matters.
Again, sticking with the Salesforce open pipeline analysis example, the most common chart we see used to explain the difference between the two points in time is a waterfall chart. It typically looks something like this:
However, we have yet to find an organization that correctly calculates this waterfall chart using the true Evolutionary Analysis principles. Even large (multi-billion dollar) enterprise organizations calculate this on a Source to End basis instead of Evolutionary. Unfortunately for most sales & marketing teams, this is a critical (sometimes fatal) miscalculation.
The Impact to Your Company
It's an obvious mistake to compare airplane directions with walking directions, but what is the impact to organizations that do the same thing with their analytics? Let's explore a real-world example comparing the difference between a Source to End and an Evolutionary analysis run on the same (fictitious) company.
While the starting and ending values are the same, the increases and decreases are much greater using the Evolutionary analysis techniques. The first chart (left) provides viewers with the illusion that they understand how their open pipeline has changed over the time periods selected, but the second chart makes it clear that only a fraction of the data is represented.
I don't know a single company that would call a discrepancy of this magnitude insignificant. The fact that the starting and ending values are the same is what makes the Source to End analysis so dangerous - it provides us with a false sense of understanding. But what's missing is insight into all of the changes that occurred between the two points in time.
Using the example above, if the company spent $10M in sales and marketing to generate pipeline, there's a huge difference between $0.68 in S&M spend to generate $1 in new pipeline and the actual amount of $0.27. And losing 27% of deals is a big difference compared to 50%. The Evolutionary analysis (second chart) makes it clear that sales and marketing is generating pipeline efficiently, but our sales funnel is a leaky bucket and we need to perform a root cause loss reason analysis to figure out why we are losing 50% of our pipeline.
Imagine the cost to this organization if executives looked at the first chart and decided to pivot marketing efforts, when the challenge might be a simple change to pricing or the sales process.
This is a simplified example, but we see organizations making their most important strategic decisions based on flawed analytical techniques due to a lack of tools and capabilities. Companies will spend millions on an ERP system, and then acquire a company, perform mass layoffs, slash budgets, and ax products based on the flawed analysis performed in a free tool. We believe that the best companies will leverage a Decision Science Platform to perform more advanced analysis, like Evolutionary analysis, to make data-driven decisions and stay ahead of the competition.
