OTLP tax under load

What does OTLP metrics push cost versus observability off under forward_fast?

Numbers are same-host comparisons on a single reference host and are not service-level objectives. See the performance hub disclaimer.

When this matters

Operators enabling OTLP metrics push need a directional sense of hot-path cost versus leaving metrics off, and how that compares to a Prometheus scrape posture. Configure push under metrics.otel. This study reuses existing feature_tax cells; the OTLP member requires the conduit-otlp-metrics-tracer companion and is skip-tolerant in the harness. Compare also the metrics scrape tax.

What we varied

Varied: observability posture (metrics_off, metrics_otlp_push, metrics_standard_scrape)
Held fixed: sync runtime, forward_fast, same dnsperf recipe on the single reference host
Skip rule: if the OTLP companion is unavailable, the OTLP pole is omitted or marked unavailable — figures never invent series

Evidence

Feature tax — OTLP push vs baselines (forward_fast)

Download CSV

Posture	Runtime	Achieved QPS	Avg latency (ms)	Sent	Completed	Workers
metrics_off	sync	75940.7	26.3	761386	761386	ingress=2
metrics_otlp_push	sync	70315.7	28.4	705512	705512	ingress=2
metrics_standard_scrape	sync	70029.9	28.5	703153	703153	ingress=2

At a glance

OTLP push vs baselines (forward_fast): metrics_otlp_push costs about 7% QPS versus metrics_off (~70k vs ~76k); metrics_standard_scrape costs about 8% QPS versus metrics_off (~70k vs ~76k).

Takeaway

OTLP push costs about as much as standard scrape on this median. Versus observability off (~76k): OTLP about 8% (~70k), standard scrape about 8% (~70k).

What to do: enable OTLP only when you have a collector path. Otherwise prefer scrape and size from the metrics scrape tax.