Skip to main content

Excessive Metric Datapoint Rate

Normal Signal: Metrics

Problem

Some metric series are exported at an unusually high cadence: several datapoints per minute, sustained. High resolution has legitimate uses, but it’s often just a default export interval nobody revisited, and every extra datapoint costs ingest and storage whether or not the value changed.

Insights flags a series that sustains more than 4 datapoints per minute across two consecutive 5-minute windows. The insight reports the cadence; whether the resolution is worth it is your call.

Impact

  • Datapoint-based pricing multiplies directly with cadence.
  • Storage and query load grow with resolution most dashboards never zoom into.
  • The habit compounds: one default interval applied fleet-wide is a large standing cost.

Example

Running Insights on our own Collector found otelcol_exporter_queue_capacity emitting 6 datapoints per minute across 18 series with a 0% change rate. The value literally never moved. Dropping it, or exporting it once a minute, carries the same information at a sixth of the datapoints.

How to fix

  1. Check how often the value meaningfully changes; match the export interval to that, not to the SDK default.
  2. Lengthen the export or scrape interval for slow-moving gauges; 30 or 60 seconds is enough for most.
  3. For values that almost never change, consider dropping the series and alerting on the config instead.

Expected gains

Datapoint volume falls with no visible difference on dashboards, and the cadence you do pay for is a decision instead of a default.

See the Insights catalog for everything Insights can detect.