kyma-project / kyma

Kyma is an opinionated set of Kubernetes-based modular building blocks, including all necessary capabilities to develop and run enterprise-grade cloud-native applications.
https://kyma-project.io
Apache License 2.0
1.52k stars 405 forks source link

Cover the `sampled` header in context of tracing in function runtimes #13423

Closed pPrecel closed 2 years ago

pPrecel commented 2 years ago

Description

After finishing this story add functionality to configure tracer (or tracer context) to record only spans when TraceID would be recorded. There are the following example headers coming together with a request:

"x-b3-traceid": "907be1c761e4cd9265d3387e341d8700",
"x-b3-spanid": "a4146c9faad5fb02",
"x-b3-parentspanid": "468ff3491ae7bb61",
"x-b3-sampled": "1"

There is a header x-b3-sampled which means if is this call recorded or not. If this value is set to 1 then we should record all spans around this request and if not then we should only allow to create spans but not send them to the jaeger service.

Reasons

This proposition is logged to be consistent with the whole istio-mesh which is reacting to this header and to keep the configuration of the functions sampling in one place together with the istio

kwiatekus commented 2 years ago

@dbadura There is a problem with proper handling of the b3-sampled header in case axios is required in the user code. I.e

require('axios')
module.exports = { 
  main: function (event, context) {
    console.log("Sampled?", event.extensions.request.headers['x-b3-sampled'])
    return "Hello World!";
  }
}

For the above function that traces are still pushed to jaeger backend even in case a request is marked as not sampled:

Screenshot 2022-03-25 at 14 31 28

Screenshot 2022-03-25 at 14 31 09

It looks like our auto-intrumentation again has problems with dependency management between runtime and user module.

kwiatekus commented 2 years ago

There are two following traces in jeager. First trace (chronologically) corresponds to a request with sampled enforced - we can observer 2 spans. One belonging to function, second from istio gateway Second trace corresponds to a request w/o b3-sampled header - I'd expect that trace information is not pushed to jeager - but it is - a single span belonging to function execution Screenshot 2022-03-25 at 15 32 20