Oh no, another Slack message from a stakeholder… “Why doesn’t the total MRR for last year in Tableau match the total MRR for last year in Hubspot?” Messages like this are every data analyst and analytics engineer’s worst nightmare as it means it’s time to pull up your sleeves and investigate your data lineage.
More often than not the same metric is defined differently in two (or more) places within your data stack. This requires you to explain to stakeholders that they don’t match because of two different definitions, compounding the lack of trust that your stakeholders already have in the data.
While this situation is a nightmare many data people know all too well, it has a simple solution: The semantic layer.
What is a universal semantic layer?
Semantic layers are your source of truth for metric calculations. They allow you to define a metric in one place without worrying about all the issues that come with the same metric in multiple places. Semantic layers prevent conflicting definitions within the same tool and across different tools. They make it easy to update a metric calculation in one place and have that definition trickle into the BI layer.
For a deeper dive into understanding how semantic layers work, check out this article.
Why semantic layers matter?
Metrics exist to help stakeholders make the most accurate decisions for the business based on the numbers they see. Without a reliable way to get the correct data, these decisions are useless. Because semantic layers centralize metrics, making them easier for data teams to update and keep track of, they reduce the likelihood of stakeholders acting on the wrong numbers. Semantic layers create more reliable, consistent, and flexible data, helping increase stakeholders’ trust in the numbers and therefore make more data-driven decisions.
3 signs a metric should be added to the semantic layer
Just like any data tool, semantic layers can be poorly implemented in a data stack, causing more of a mess than anything else. We want to avoid dumping all metrics trapped in BI tools like Looker and Tableau into our semantic layer. Rather than a simple lift and shift, we need to think about it strategically.
If your metric meets these three standards, it’s safe to say that it should be moved to the semantic layer:
1. It’s used by multiple teams
How many times have you added the same metric to multiple dashboards? The growth team wants to see revenue in their dashboard but finance also needs to see it in theirs. Before you know it, this metric is sprawled across many different dashboards, used by many different teams, and you’ve lost track of what is where.
Metric calculations added to specific team dashboards in tools like Looker and Tableau tend to get lost. This then creates metric drift; slowly over time, the definitions of each metric slightly change. Before you know it, every team uses a different calculation for the same metric, creating confusion and decisions that don’t align.
Metrics used by multiple teams need to be centralized in the semantic layer so you can update definitions once and have them trickle into individual teams’ dashboards.
2. It’s a metric the business will care about 3 years from now
If you take the time to add a metric to the semantic layer, it needs to be one you plan to use for a while. Think about the semantic layer as building a long-term foundation for high-quality data in your BI tool. Just as you don’t save ad-hoc queries as data models, you don’t need to save one-time metrics in the semantic layer.
Metrics used to track the success of one-off projects and experiments don’t need to be centralized, as it’s often more work than it’s worth. These are likely only used by one or two teams in the short term. Instead, focus on moving “north star” metrics to the semantic layer. These are the metrics that have the power to move the needle for the entire business. Improving upon their definition and defining them in a way that’s measurable and scalable in the long term will have infinite returns for the business.
3. Its calculation is complex and often confuses those who use it
The semantic layer allows you to more easily define metrics and their various dimensions. It acts as a blueprint on how to use the company’s data models to calculate metrics, giving insight into the different ways you can slice and dice the data. They document qualitative and quantitative dimensions of a metric, including time series data like dates and timestamps. This is particularly helpful for those less familiar with the data, but still want to feel empowered to use it.
Semantic layers clear up any confusion on how a metric is calculated, giving them the confidence to take data-driven decisions into their own hands.
Metrics to centralize in your semantic layer
There are a few metrics that come to mind when I think of the most important ones to move to a semantic layer. Of course, these will vary depending on the business, but for most, these check the above boxes.
Lifetime value (LTV)
LTV will always be a metric that the business cares about. LTV relates directly to revenue and how valuable certain cohorts of customers are towards the health of the business. For many businesses, it’s the north star metric that all efforts lead back to.
LTV is a perfect example of a metric that every team closely measures. Sales cares about the value of the customers they are bringing in through its leads. Marketing needs to know if their blog posts drive long-term customers to the business so they can decide where to continue their efforts. Product is interested to see how their new features are affecting expansion.
Because LTV is a metric touched by so many teams, it’s important that it’s centralized in the semantic layer. This ensures all teams use the same LTV calculation to drive their business strategy, leaving little room for conflicting numbers.
Customer acquisition cost (CAC)
CAC is a metric business will always care about as it’s critical to how marketing dollars are spent. As long as marketing is spending money on customers, they will want to know how much they are spending to acquire a customer and how it relates to revenue metrics like LTV.
When working for a wine subscription company, our marketing team constantly relied on CAC to understand which marketing channels were most successful for the business. The channels with the lowest CAC and highest retention rates were prioritized in terms of marketing spend. If the CAC was high and the LTV low, the channel wasn’t worth putting more money into.
Because this metric was a north star metric for our marketing team, it made sense to move it to a semantic layer. Moving it to a semantic layer ensured its calculation could be tracked over time with little room for errors like metric drift. In addition, new dimensions could easily be added as data models increased their scope and new requirements came up. Bringing it to the semantic layer was worth all of the time and effort because of how vital the metric was to the long-term success of the marketing team.
Count of active users
This is a metric I’ve seen cause a lot of confusion across different teams and companies. It seems like every team has a different idea of what “active” means for them. For engineers, this could be based on the last time a user visited the website while product defines it as the last time a user used a certain feature.
The count of active users is a great metric to bring to the semantic layer because it forces various teams to come together and talk about a metric that lacks clarity. When you bring a metric to the semantic layer, you’re forced to discuss a shared definition of the metric. It allows for conversations that many people are otherwise too busy to have.
For many companies, bringing confusing metrics like the count of active users to the semantic layers allows for autonomy and clarity that never before existed. It helps them bridge the disconnect between the success benchmarks of teams. Bringing shared metrics to the semantic layer allows for a true single source of truth that can empower everyone to feel confident in the numbers they are seeing.
Not to mention, when someone wants to adjust a metric definition, they are forced to discuss it with all team leaders rather than making the change in their own siloed area of the business. Talk about avoiding metric drift!
What NOT to add to the semantic layer
1. Metrics from one-time experiments, exploratory work, or testing
Metrics that are core to the business and used by multiple teams should always be added to your semantic layer. However, metrics used for one-time experiments, exploratory work, and testing don’t make sense in the semantic layer. The semantic layer is for metrics that are well-fleshed out, not those that are only starting to be understood. Until they become metrics that belong in production and measure a concept foundational to how the business operates, they can stay defined in your BI tools.
Metrics used to measure the success of A/B tests are a great example of this. They are helpful for a certain period, or for a specific team, but won’t last beyond the experiment. Define them within your BI tools, use them to guide the results of the experiment, and be done with them.
2. Metrics rarely used
If a metric is rarely used, don’t waste time moving it to the semantic layer. Remember, metrics in the semantic layer stand the test of time and are used by multiple teams within a business. If a metric is rarely used, it is less likely to experience metric drift or lead to a lack of trust across the business. Instead, focus your attention on metrics that drive the business!
Metrics related to events that happen once a year fit in this category. Think of metrics that measure the success of Black Friday sales for example. They are crucial to how the business handles sales, but they only get looked at once each year. Instead, take your time moving metrics that are used all year round.
3. Immature metrics
When a metric is first created, the definition rarely sticks. One team pioneers the original metric and then other teams begin to weigh in. The metric constantly changes depending on what the company learns and how it changes the features that the metric measures. It’s best to wait until the business knows how the metric fits into the business as a whole and is better understood by all teams.
An immature metric would be a success metric created for a new product or feature launched within the business. Oftentimes, the business doesn’t necessarily know what success looks like for the product or feature until it understands how its users are using it. Because of this, the metric changes over time as the business understands more about user behavior.
Making sense of your metrics
Gathering all of the metrics in your BI tools to audit for the semantic layer can seem like a daunting task. Luckily, Euno helps you gather all the metrics locked within BI, identifying lineage to the data layer, field-level utilization, and inconsistencies in metrics. It then helps you to shift these metrics left, establishing a source of truth. With Euno automatically governing metrics in your BI tool, you can spend more time having important conversations with stakeholders and less on the nitty-gritty work of finding hidden metrics.
Conclusion
Semantic layers help avoid the most common (and costly) mistakes made by data teams. They increase reliability, flexibility, and consistency. When thinking about which metrics to move over to the semantic layer, ask yourself:
- What teams use this metric?
- Will this metric matter to the business 3 years from now?
- Do stakeholders often come to me with questions about how to use this metric?
Euno makes it easy to help you answer these questions, mapping out your BI logic. The answers given to you by Euno help guide you on which metrics to move away from BI tools like Looker and Tableau and to a more universal, central solution like the semantic layer.
