set upstream

curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -i -d '
{
    "uri": "/*",
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "192.168.8.109:8200": 10
        }
    }
}'

set opentelemetry

curl http://127.0.0.1:9080/apisix/admin/global_rules/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -i -d '
{
    "plugins": {
        "request-id": {
            "algorithm": "range_id",
            "header_name": "X-Request-Id",
            "include_in_response": true
        },
        "opentelemetry": {
            "sampler": {
                "name": "trace_id_ratio",
                "options": {
                    "fraction": 0.5
                }
            }
        }
    }
}'

send request with curl

curl http://127.0.0.1:9080/vv 100 times

Expected Behavior

The plugin should have a 50% chance of sending a trace, but in reality, no trace was sent, and I saw that there was no testing for this part in the test case

Error Logs

No response

Steps to Reproduce

as Current Behavior description

Environment

APISIX version (run apisix version): master branch
Operating system (run uname -a): Linux myapisixnode 3.10.0-1160.el7.x86_64 #1 SMP Mon Oct 19 16:18:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

OpenResty / Nginx version (run openresty -V or nginx -V):

built by gcc 8.3.1 20190311 (Red Hat 8.3.1-3) (GCC)
built with OpenSSL 1.1.1s  1 Nov 2022
TLS SNI support enabled
configure arguments: --prefix=/usr/local/openresty/nginx --with-cc-opt='-O2 -DAPISIX_BASE_VER=1.0.0 -DNGX_GRPC_CLI_ENGINE_PATH=/usr/local/openresty/libgrpc_engine.so -DNGX_HTTP_GRPC_CLI_ENGINE_PATH=/usr/local/openresty/libgrpc_engine.so -DNGX_LUA_ABORT_AT_PANIC -I/usr/local/openresty/zlib/include -I/usr/local/openresty/pcre/include -I/usr/local/openresty/openssl111/include' --add-module=../ngx_devel_kit-0.3.2 --add-module=../echo-nginx-module-0.63 --add-module=../xss-nginx-module-0.06 --add-module=../ngx_coolkit-0.2 --add-module=../set-misc-nginx-module-0.33 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.09 --add-module=../srcache-nginx-module-0.33 --add-module=../ngx_lua-0.10.25 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.34 --add-module=../array-var-nginx-module-0.06 --add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15 --add-module=../redis-nginx-module-0.3.9 --add-module=../ngx_stream_lua-0.0.13 --with-ld-opt='-Wl,-rpath,/usr/local/openresty/luajit/lib -Wl,-rpath,/usr/local/openresty/wasmtime-c-api/lib -L/usr/local/openresty/zlib/lib -L/usr/local/openresty/pcre/lib -L/usr/local/openresty/openssl111/lib -Wl,-rpath,/usr/local/openresty/zlib/lib:/usr/local/openresty/pcre/lib:/usr/local/openresty/openssl111/lib' --add-module=/tmp/tmp-new/openresty-1.21.4.2/../mod_dubbo-1.0.2 --add-module=/tmp/tmp-new/openresty-1.21.4.2/../ngx_multi_upstream_module-1.1.1 --add-module=/tmp/tmp-new/openresty-1.21.4.2/../apisix-nginx-module-1.15.0 --add-module=/tmp/tmp-new/openresty-1.21.4.2/../apisix-nginx-module-1.15.0/src/stream --add-module=/tmp/tmp-new/openresty-1.21.4.2/../apisix-nginx-module-1.15.0/src/meta --add-module=/tmp/tmp-new/openresty-1.21.4.2/../wasm-nginx-module-0.6.5 --add-module=/tmp/tmp-new/openresty-1.21.4.2/../lua-var-nginx-module-v0.5.3 --add-module=/tmp/tmp-new/openresty-1.21.4.2/../grpc-client-nginx-module-v0.4.4 --add-module=/tmp/tmp-new/openresty-1.21.4.2/../lua-resty-events-0.2.0 --with-poll_module --with-pcre-jit --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_v2_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_auth_request_module --with-http_secure_link_module --with-http_random_index_module --with-http_gzip_static_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-threads --with-compat --with-stream --with-http_ssl_module

etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info): 3.5.10
APISIX Dashboard version, if relevant:
Plugin runner version, for issues related to plugin runners:
LuaRocks version, for installation issues (run luarocks --version):

wklken commented 1 week ago

You can remove the plugin request-id then try again.

While the default trace_id_source of opentelemetry plugin is x-request-id, if the x-request-id is not a valid traceID, it would not been reported.

the doc: https://apisix.apache.org/docs/apisix/plugins/opentelemetry/#configuring-the-collector

zhendongcmss commented 1 week ago

I removed request-id plugin then try curl http://127.0.0.1:9080/123 -H "X-Request-Id: 1272b56c15a7866668b943071c176805" trace_id_ratio doesn't work too. apisix always send the trace.

If curl without request-id, trace_id_ratio it can work.

wklken commented 1 week ago

There is a sampled flag in the trace id, the opentelemetry plugin will respect the flag.

Maybe you should check the source code here https://github.com/yangxikun/opentelemetry-lua/blob/main/lib/opentelemetry/trace/sampling/trace_id_ratio_sampler.lua#L34

apache / apisix

bug: opentelemetry `trace_id_ratio` doesn't work #11752

Current Behavior