otel_span_attr directive does not work on location directive

[alquerci]

Documentation used

https://nginx.org/en/docs/ngx_otel_module.html#otel_span_attr

Environment

Include the result of the following commands:

• docker run --rm nginx:1.27.1-alpine3.20-otel nginx -V

nginx version: nginx/1.27.1
built by gcc 13.2.1 20240309 (Alpine 13.2.1_git20240309) 
built with OpenSSL 3.3.0 9 Apr 2024 (running with OpenSSL 3.3.2 3 Sep 2024)
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --with-perl_modules_path=/usr/lib/perl5/vendor_perl --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-http_v3_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-Os -fstack-clash-protection -Wformat -Werror=format-security -fno-plt -g' --with-ld-opt='-Wl,--as-needed,-O1,--sort-common -Wl,-z,pack-relative-relocs'

• docker run --rm nginx:1.27.1-alpine3.20-otel uname -a

Linux c9a1f0f6ef66 5.15.0-122-generic #132~20.04.1-Ubuntu SMP Fri Aug 30 15:50:07 UTC 2024 x86_64 Linux

Description

otel_span_attr directive is used in location directive is same as if otel_span_attr directive is used in server directive.

• [x] The bug is reproducible with the latest version of nginx
• [x] The nginx configuration is minimized to the smallest possible to reproduce the issue and doesn't contain third-party modules

nginx configuration

load_module modules/ngx_otel_module.so;

http {
  otel_exporter {
    endpoint "collector:4317";
  }

  server {
    location / {
      otel_trace on;
      otel_trace_context propagate;
      otel_span_attr deployment.environment.name "dev";

      try_files $uri /index.php/$request_uri;
    }

    location ~ ^/index\.php(/|$) {
      fastcgi_pass php:9000;
    }
  }
}

or share the configuration in gist.

nginx debug log

It is advised to enable debug logging.

# Your nginx debug log here

or share the debug log in gist.

Workaround

Move location directives on server.

  # ...
  server {
    otel_trace on;
    otel_trace_context propagate;
    otel_span_attr deployment.environment.name "dev";

    location / {
      try_files $uri /index.php/$request_uri;
    }
    # ...

Initially posted https://github.com/nginx/nginx/issues/195

[p-pautov]

The attributes are applied from the final location of the request. Looks like, try_files redirects your request to the regexp location, which doesn't have any otel_span_attr directives.

[alquerci]

Hello @p-pautov and thanks for the fast reply.

Let's try another alternative then

load_module modules/ngx_otel_module.so;

http {
  otel_exporter {
    endpoint "collector:4317";
  }

  server {
    location / {
      try_files $uri /index.php/$request_uri;
    }

    location ~ ^/index\.php(/|$) {
      otel_trace on;
      otel_trace_context propagate;
      otel_span_attr deployment.environment.name "dev";

      fastcgi_pass php:9000;
    }
  }
}

Result

No trace sent.

[p-pautov]

That's because the sampling/propagation decision is made in the initial request location. We have #55 dedicated to this.