SpanMetrics connector: turning traces into RED metrics

There is a very normal moment after a team starts sending traces. The traces are useful. You can open one request, follow it through services, and see where time went.

Then someone asks a simpler question: is checkout slower than usual?

Opening individual traces is the wrong shape of work for that. You want a graph. You want request volume, error rate, and latency over time. The funny part is that the data needed for those metrics is already sitting inside the spans. A server span knows which service emitted it, what operation it represents, when it started, when it ended, and whether it failed.

The span_metrics connector sits in that exact gap. It turns span activity into metrics inside the OpenTelemetry Collector, while still letting traces remain traces. Metrics tell you something changed. Traces explain the request path behind the change.

Why traces alone are not enough

Traces are for depth. They are brilliant when you need to understand one request.

Dashboards and alerts need a different shape. They need stable time series:

request count by service and route
error count by service and route
p95 latency by service and route

You can emit those metrics directly from application code, and for business concepts you often should. orders_created, payments_authorized, and tokens_generated should not be reverse engineered from spans if the application already knows the business event.

But RED metrics are different. Rate, errors, and duration are often implied by server spans. If every service already emits traces, maintaining a second set of hand-written request metrics can become busywork.

Span-derived metrics are not a replacement for instrumentation. They are a reuse of trace data for the dashboard layer.

What the connector creates

The current Collector component is span_metrics. Older examples may still show spanmetrics, but new configs should use span_metrics.

The connector sits between a traces pipeline and a metrics pipeline:

traces pipeline -> span_metrics -> metrics pipeline

That sentence sounds strange until the Collector model clicks. The connector behaves like an exporter from the traces side and a receiver on the metrics side. Spans enter. Metrics leave.

The useful outputs are:

traces.span.metrics.calls
traces.span.metrics.duration

calls counts spans by dimensions. duration creates a histogram from span start and end time. Error information usually comes through a status dimension, which means an error span contributes to the same metric family with a different status.

Common dimensions include service.name, span.name, span.kind, and status.code. You can add dimensions such as http.request.method, http.route, rpc.service, or db.system.name.

That is where discipline matters. These are metrics now. If you add raw URLs, user IDs, pod names, tenant IDs, or request IDs as dimensions, you have turned a helpful connector into a cardinality generator.

The pipeline shape

The clean setup sends traces to a trace backend and generated metrics to a metrics backend. They describe the same traffic, but they are not the same signal.

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:

connectors:
  span_metrics:
    namespace: traces.span.metrics
    histogram:
      unit: ms
      explicit:
        buckets: [5ms, 10ms, 25ms, 50ms, 100ms, 250ms, 500ms, 1s, 2s, 5s]
    dimensions:
      - name: http.request.method
      - name: http.route
      - name: rpc.service
      - name: db.system.name
    aggregation_cardinality_limit: 10000
    metrics_flush_interval: 30s
    series_expiration: 10m

exporters:
  otlp/traces:
    endpoint: traces.example:4317
  otlp/metrics:
    endpoint: metrics.example:4317

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [span_metrics, otlp/traces]
    metrics/spanmetrics:
      receivers: [span_metrics]
      processors: [batch]
      exporters: [otlp/metrics]

Read the config from the pipelines. The traces pipeline sends spans to both the trace backend and the connector. The metrics pipeline receives generated metrics from the connector and exports them.

That separation is the point. The generated metric should not be treated as a trace. The trace should not be discarded just because a metric exists.

Cardinality is the real design problem

The connector is easy to turn on. The hard part is deciding which dimensions deserve to become metric labels.

Good dimensions are stable and bounded:

Dimension	Usually safe?	Why
service.name	Yes	Services are the first useful grouping
span.kind	Yes	Small, well-known value set
status.code	Yes	Needed for error rate
http.request.method	Yes	Small set of methods
http.route	Usually	Good if routes are templated
http.target	No	Often contains raw paths and query strings
user.id	No	High cardinality and often sensitive
tenant.id	Be careful	Useful, but needs limits and retention rules

The difference between http.route="/users/{id}" and http.target="/users/12345?debug=true" is the difference between useful request metrics and a time-series explosion.

The connector does not remove the need for naming discipline. It makes discipline more important because trace attributes can become metric dimensions.

What to alert on

Once the connector is in place, the generated metrics let you build the classic request dashboard:

Question	Metric shape
How many requests are flowing?	Count calls by service and route
Are errors increasing?	Filter or group calls by error status
Is latency changing?	Use the duration histogram
Which route is noisy?	Group by http.route
Which dependency is slow?	Group client spans by db.system.name or rpc.service

Keep the alerting layer boring. Alert on service-level symptoms first. Use route or dependency dimensions for triage. Do not start with every possible label in the page.

Metrics should get someone to the right neighborhood. Traces should explain the house.

How to validate it

Start small. One service, one route, one Collector, one metrics backend. Generate traffic, then check that:

1. Spans still reach the trace backend.
2. Generated metrics reach the metrics backend.
3. Labels are stable and bounded.
4. Histograms have sensible buckets.
5. The number of time series matches your expectations.

Then test failure. Return a 500. Make sure the status dimension changes. Slow a request down. Make sure the duration histogram moves.

If the metric exists but the route label is missing, look at the span attributes. If the route label exists but cardinality jumps, check whether you used a templated route or a raw path. If nothing shows up, add a debug exporter to the relevant pipeline and prove whether the spans reach the connector.

Conclusion

The SpanMetrics connector is useful because it respects the different jobs of traces and metrics.

Traces are still the high-resolution story of a request. Metrics are still the compact view you can alert and graph. The connector lets one signal help produce the other without pretending they are the same thing.

Used carefully, it gives teams RED metrics with less duplicated instrumentation. Used casually, it can create too many time series from span attributes that were never meant to be metric labels.

The whole game is picking dimensions like you will have to operate them later, because you will.