Scrub PII at the source with the OpenTelemetry Collector

PII does not politely stay out of telemetry.

A log line can carry an email address. A span attribute can hold a full URL with query parameters. A database statement can include customer identifiers. A request header can contain a token. A GenAI trace can contain prompt text, retrieved documents, or tool arguments.

By the time that data lands in an observability backend, you are already doing cleanup. The safer move is to prevent the raw value from leaving your environment.

That is why source-side scrubbing matters. If the OpenTelemetry Collector removes, masks, or hashes sensitive fields before export, every downstream system only sees the cleaned version.

The Collector is the right control point

The Collector already sits in the middle of the path:

application or SDK
  -> Collector receiver
  -> processor
  -> exporter
  -> observability backend

Processors run before exporters. That ordering matters. If the processor deletes an authorization header before the exporter runs, the backend never receives the original header.

You are not relying on a retention policy. You are not hoping someone remembers to avoid a dashboard field. You are changing the data before it leaves.

The risky fields are usually familiar:

Risky area	Examples
Identity	user.email, user.id, customer.id
Session data	session.id, cookies, auth headers
URLs	Raw query strings, invite links, reset tokens
Database	SQL statements, bind parameters
GenAI	Prompts, retrieved documents, tool arguments
Custom fields	Anything a team added because it helped debugging once

The problem is not that engineers do not know these fields are sensitive. The problem is that telemetry pipelines make it very easy for useful context to become retained context.

Pattern one: known fields

When the schema is stable, use direct attribute actions. This is the simplest and most auditable pattern.

processors:
  attributes/scrub_known_fields:
    actions:
      - key: http.request.header.authorization
        action: delete
      - key: user.email
        action: hash
      - key: session.id
        action: delete

This works well when your platform has a naming contract. If every service uses user.email, the rule is boring. Boring is good here. Boring can be reviewed.

Use this for fields where the key itself is the policy.

Pattern two: unpredictable keys and values

Sometimes the schema is not stable. One service writes api_token, another writes access_token, and another hides an email in a free-form log attribute.

That is where a redaction-style policy helps. You can run it as an allowlist:

processors:
  redaction/allowlist:
    allow_all_keys: false
    allowed_keys:
      - service.name
      - deployment.environment.name
      - http.request.method
      - http.route
      - http.response.status_code

Or you can keep most fields while blocking suspicious keys and values:

processors:
  redaction/mask_patterns:
    allow_all_keys: true
    redact_all_types: true
    blocked_key_patterns:
      - ".*token.*"
      - ".*password.*"
      - ".*secret.*"
    blocked_values:
      - "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}"

Allowlists are safer. Blocklists are easier to adopt. Most teams end up with both: strict defaults for shared platform fields, and pattern-based protection around the messy edge.

Pattern three: context-aware transforms

The transform processor is useful when the rule depends on signal type, field location, or a pattern inside a value.

processors:
  transform/scrub_sensitive:
    error_mode: ignore
    trace_statements:
      - delete_key(span.attributes, "http.request.header.authorization")
      - replace_pattern(span.attributes["process.command_line"], "password\\=[^\\s]*(\\s?)", "password=***")
    log_statements:
      - delete_key(log.attributes, "http.request.header.authorization")
      - replace_pattern(log.body, "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}", "***@***")

This is the flexible option. It is also the one that deserves careful tests because flexible rules are easier to misunderstand.

Use transform rules when you need logic. Use direct attributes when you already know the key.

Do not only clean logs

PII is not only a logs problem.

Traces can contain request headers, SQL statements, route parameters, prompt payloads, model inputs, tool arguments, and custom span attributes. Metrics can carry risky labels if teams put user or tenant identifiers in dimensions.

Scrubbing policy should be signal-aware:

Signal	Common leak
Logs	Message bodies, structured fields, stack traces
Traces	Span attributes, headers, SQL, URLs, prompts
Metrics	High-cardinality labels and user identifiers

For metrics, the right answer is often “do not emit that label” rather than “scrub it later.” Once a sensitive value becomes a metric label, it can create high-cardinality time series and spread through aggregation paths.

Pipeline order matters

Run scrubbing before export and before any processor that fans data out to multiple destinations.

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [k8s_attributes, attributes/scrub_known_fields, transform/scrub_sensitive, batch]
      exporters: [otlp/backend]

The exact order depends on your pipeline, but the intent is simple:

1. Add metadata needed for routing and policy.
2. Scrub sensitive content.
3. Batch and export cleaned data.

If one exporter goes to a production backend and another goes to a debugging sink, both should receive the cleaned version.

Test with fake secrets

Do not deploy privacy rules because the YAML looks correct.

Create a test record with fake sensitive values:

user.email = "test@example.com"
http.request.header.authorization = "Bearer fake-token"
process.command_line = "run --password=secret"

Run it through the Collector with a debug exporter after the scrub processors. Confirm the output is deleted, hashed, or masked as expected.

Then test the negative case. Make sure safe attributes such as service.name, http.route, and deployment.environment.name survive. A privacy policy that deletes all useful context will be bypassed by frustrated engineers later.

Governance is part of the pipeline

Scrubbing is not only a YAML problem. It is a product decision for your observability platform.

Define:

Policy area	Decision to make
Known sensitive keys	Delete, hash, or mask
Raw content capture	Default off or explicit opt-in
Domain identifiers	Allowed in logs, spans, metrics, or none
GenAI content	Redact, truncate, sample, or drop
Review process	Who approves new high-risk fields

Good privacy controls do not make debugging impossible. They make the safe path obvious and the risky path explicit.

Conclusion

The safest observability backend is the one that never receives raw secrets in the first place.

The OpenTelemetry Collector gives you the control point to make that happen. Use direct attribute actions for known keys, redaction rules for messy schemas, transform rules for context-aware cleanup, and debug output to test the result.

Scrub before export. Apply the policy to logs and traces, not only one signal. Keep metrics labels clean by design.

Privacy is much easier when the sensitive value never leaves your network.