Skip to main content

High-Cardinality Metric Attribute

Important Signal: Metrics

Problem

One of your metrics carries an attribute with unbounded values: user IDs, session IDs, full URLs, raw SQL. Every distinct value creates a separate time series, so a single label can quietly turn one metric into millions of series.

Insights tracks the distinct values seen per metric attribute and flags an attribute once it passes 1,000 distinct values.

Impact

  • Storage and cost grow with every new value, not with your traffic.
  • Dashboards slow down: every query scans millions of series to draw one line.
  • Alerting stops making sense; you can’t alert per user ID.

Example

Exploding:

# metric http_requests_total, attributes
user_id: "9f2a41c7" # a new series per user
session_id: "d8ab99e1" # a new series per session
url: "/product/12345?ref=email" # a new series per URL

Stable:

# metric http_requests_total, attributes
service_name: "checkout"
http_route: "/product/{id}"
http_method: "GET"

How to fix

  1. Remove the flagged attribute from the metric, or replace it with a bounded version: the route template instead of the URL, the plan tier instead of the customer ID.
  2. If you need per-entity detail, put it on spans or logs, which handle high cardinality far better than metric labels.
  3. If you can’t change the code, drop or rewrite the attribute with a Collector processor before export.

Expected gains

Series counts and storage stabilize regardless of traffic, dashboards get fast again, and alerts describe services instead of individual users.

See the Insights catalog for everything Insights can detect.