Access Span and Trace ID in Lua Script

[plechi]

I am using it with ingress-nginx (v 1.9.5) and have it set up like in the documentation: https://kubernetes.github.io/ingress-nginx/user-guide/third-party-addons/opentelemetry/

Unfortunately, to correlate the logs and traces with datadog, one needs to transform the log and span ids like described here:

https://docs.datadoghq.com/tracing/other_telemetry/connect_logs_and_traces/opentelemetry/?tab=python

So I created a lua plugin and use the log_by_lua block. However, the nginx variables are not set.

My code is:

-- simplified, it's actually called by the ingress-nginx plugin system, but the plugin is called from within log_by_lua
log_by_lua_block {
    local trace_id = ngx.var.opentelemetry_trace_id
    local span_id = ngx.var.opentelemetry_span_id
    -- do the transformation according to the datadog documentation
    ngx.var.trace_id=trace_id
    ngx.var.span_id=span_id
}

I then access it in my logs via the $span_id and $trace_id variables.

However, ngx.var.opentelemetry_trace_id and ngx.var.opentelemetry_span_id is always nil. If I manually set the variables like local trace_id='somevalue' it prints the correct value in the log.

• is log_by_lua_block the correct place to get the values? When are they actually set by the plugin?
  • is ngx.var.opentelemetry_trace_id the place where I can find the variable? I also looked at ngx.ctx but couldn't find anything there.

[stoetti]

Did you find any solution how to attach the trace- and span-id in the logs?

[johanneswuerbach]

We are using a custom log format with the following and logs are correlated in datadog.

controller:
  config:
    log-format-escape-json: true
    log-format-upstream: >-
      {
      ... various field ...
      "dd.trace_id": "$opentelemetry_trace_id",
      "dd.span_id": "$opentelemetry_span_id"
      }