traceparent header doesn't get propagated from golang applications when its added as a sidecar in kubernetes

[lokesh411]

Hey I have written an API in golang (Service A using gofiber as http framework) which will call one of my services which is written in NodeJS (service B), the sidecar is only added to the service A. So when i hit the API of service A with a traceparent header, when an API call is made from this route, the traceparent header is not propagated Please help me understand if i have misunderstood something Thanks in advance

[grcevski]

Hi, what you describe here should've worked. Do you mind adding BEYLA_PRINT_TRACES=1 (as environment variable) or add print_traces: true in your Beyla configuration and show us the output? It will print on the standard output what server and client calls it's capturing and what the TraceIDs are.

We haven't tested with gofiber, if you have a simple example for us to try with that doesn't work, that will also help a lot.

Thanks!

[lokesh411]

Hey

beyla 2024-04-18 19:06:18.4187618 (386.604µs[386.604µs]) 200 POST /v1/traces []->[otel-trace-lb.monitoring:4318] size:563B svc=[services/profile-service go] traceparent=[00-979f230b1d3f9c1839b20877bfcc836c-0000000000000000-01]

This is the debug trace that i got in service A, the traceparent is completely different which was passed to service B Here is the example snippet which i used to test

router.Post("/test/golang", func (c *fiber.Ctx) error {
        url := "http://b-service:3000/hello"
        method := "GET"

        client := &http.Client{}
        fmt.Println("The request headers for this request is :: ", c.GetReqHeaders())
        req, err := http.NewRequest(method, url, nil)

        if err != nil {
            fmt.Println("Error in creating a request to data-service :: ", err)
            return err
        }
        res, err := client.Do(req)
        if err != nil {
            fmt.Println("Error in making a request to data-service :: ", err)
        }
        defer res.Body.Close()

        body, err := io.ReadAll(res.Body)
        if err != nil {
            fmt.Println("Error in decoding the response from the API :: ", err)
            return err
        }
        fmt.Println("The response from the health-check endpoint is :: ", string(body))

        response:= fiber.Map{
            "success": success,
            "message": message,
            "data":    data,
        }

        return c.JSON(response)
    })

[lokesh411]

This seems to be working well in echo http framework, would it be because gofiber is using the existing goroutines to serve HTTP calls?

[grcevski]

Hi again, so that looks like a regular httpclient call, we should've been able to propagate the context. From the requests output we only see a call to /v1/traces, which makes me think you are not instrumenting the correct service, unless your service has the OTel SDK added and is also making those calls?

How do you select the Go process to be instrumented with Beyla? Do you use Open Port or Executable Name?

I would've expected that we see a trace for /test/golang and one for /hello. Even if we couldn't properly instrument gofiber, the client instrumentation would've caught the call to /hello.

[lokesh411]

Yeah the service is also configured with OTel SDK so sends traces to collector, I used the open port to instrument the Go process Currently the client instrumentation doesn't propagate any incoming context to the external services (the code changes for that hasn't been done yet)

[grcevski]

OK, thanks! I'll try to create a fiber testcase and see what happens.