How it works
Compare answers one question: why did my KPI move from Start to End? It answers it in three layers.
The market equation writes the KPI as a product of steps, for example:
Revenue = Users × Sessions/User × Checkouts/Session × Revenue/Checkout
Compare measures the variation of each step between Start and End, and turns it into an impact on the KPI, in the unit of the KPI. The impacts of the steps add up to the total gap.
For example, revenue fell by 28.1%, from 45,549 € to 32,747 €. Users fell by 11.7% and checkouts per session by 26.7%; the drop of checkouts per session alone costs 12,298 €.
Dimensions and segments
Section titled “Dimensions and segments”For each step, Compare scores every dimension: how well its segments explain the variation of the step. It shows the most interesting dimension first, with the impact of each of its segments, for example the countries that explain the drop of checkouts per session.
You can change the dimension, drill into a segment, or follow the dimension hierarchy.
The Analysis method decides where the exploration starts:
- Steps then Dimensions starts from the steps of the market equation, then explains each step with the dimensions;
- Dimensions then Steps or Dimensions starts from the most interesting dimension and drills down segment by segment. Useful when the teams you report to are organised by segment, or when two steps compensate each other;
- Auto (default) computes both and shows the most interesting one.
Mix and performance effects
Section titled “Mix and performance effects”The impact of a segment on a step has two parts:
- the mix effect comes from a change in the weight of the segment. If mobile users convert less than desktop users, a week with more mobile traffic converts less overall, even if each device converts as well as before;
- the performance effect comes from a change in the segment itself, at constant weight: mobile users converting better or worse than before.
Compare shows the two effects separately when the mix is significant. See Mix & Performance effects in the visual settings.
Covariance
Section titled “Covariance”The steps of a market equation interact: when users and conversion both drop, part of the gap comes from the two moving together, and no single step explains it alone. That part is the covariance.
By default, Compare spreads it over the steps that interact. You can isolate it in a dedicated Covariance step at the end of the waterfall, and Compare warns you when it is high: a high covariance can mean that the two sides of the comparison are not comparable populations.
Inside a step, a segment can also change its weight and its ratio at the same time. Mix & Performance covariance decides where that part goes: shared between mix and performance (default), or given to one of them. See the visual settings.
Going further
Section titled “Going further”- Segments present on one side of the comparison only: out of scope segments.
- Detail a step into sub-steps: market equation › sub-steps.
- Simulate a change: What-if scenarios.