Episode #5 – How high cardinality impacts usability and cost with Alex Kehlenbeck
Learn how observability tools can be affected by high cardinality in metrics
Summary
On episode 5 of the Observe It podcast, host ダニエラ・ミャオ welcomes Alex Kehlenbeck, a Distinguished Software Engineer at Lightstep. Daniela and Alex kick off the conversation by diving deep into the current state of observability in the industry, highlighting its complexity and fragmentation. Alex shares insights into the challenges of high cardinality in metrics and how it impacts the usability and cost of observability tools
They touch on the historical reasons behind the difficulty in handling high cardinality and how advancements in OLAP (Online Analytical Processing) space can influence observability tools positively. Alex emphasizes the need for a shift in mindset within the observability industry in order to improve the user experience.
Alex sheds light on the issues related to pricing models in the observability space, discussing the reluctance of customers to embrace alternative pricing based on data bytes rather than cardinality. He also shares Lightstep’s approach to using storage systems inspired by OLAP technologies, making observability more cost-effective and performance-oriented.
The discussion extends to the role of technology in supporting high cardinality, the challenges associated with pricing models, and the slow pace of industry mindset change. Daniela and Alex explore the potential for disrupting existing pricing models and the importance of optimizing observability tools for cost and performance.
Alex concludes by emphasizing the need for smaller players in the observability space to challenge existing norms and bring about positive changes. The episode closes with a brief discussion on the underlying storage system used in observability tools and the future convergence of metrics, traces, and logs for a seamless user experience.
About Alex Kehlenbeck
Alex currently leads the systems teams at Lightstep. Prior to that he co-created Monarch, Google’s global monitoring system, and led the development of TCube, an exabyte-scale storage system for telemetry data.