search

DataDog / orchestrion

Automatic compile-time instrumentation of Go code
http://datadoghq.dev/orchestrion/
Apache License 2.0
158 stars 5 forks source link

[BUG] Consul propagation is missing #360

Open djugalov opened 3 days ago

djugalov commented 3 days ago

Version of orchestrion

v0.9.3

Describe what happened: I'd like to share some observations following the recent PoC, where we aimed to integrate Fabio with DataDog APM using Orchestrion. After installing Orchestrion, performing orchestrion pin, and deploying the application locally, we successfully saw Fabio appear in the DataDog Service Catalog. While we were able to capture traces from Fabio, the information provided was limited. Specifically, we were unable to observe traces that included spans for requests initiated to services registered in Consul and present in Fabio's routing table. Describe what you expected: Our expectations were to be able to observe traces containing requests to services registered in consul and present in fabio routing table.

Steps to reproduce the issue:

Prerequisites

  1. Install consul - https://developer.hashicorp.com/consul/docs/k8s/installation/install

  2. Have some dummy example service registered in consul

  3. Fork fabio repository

  4. Modify existing Dockerfile with the one provided below:

    
FROM golang:1.23-alpine3.20 AS build

ARG consul_version=1.19.2 ADD https://releases.hashicorp.com/consul/${consul_version}/consul_${consul_version}_linux_amd64.zip /usr/local/bin RUN cd /usr/local/bin && unzip consul_${consul_version}_linux_amd64.zip

ARG vault_version=1.17.5 ADD https://releases.hashicorp.com/vault/${vault_version}/vault_${vault_version}_linux_amd64.zip /usr/local/bin RUN cd /usr/local/bin && unzip -o vault_${vault_version}_linux_amd64.zip

RUN apk update && apk add --no-cache git libcap WORKDIR /src COPY . .

RUN go mod tidy

Install Orchestrion

RUN go install github.com/DataDog/orchestrion@latest

Register Orchestrion in the go.mod file

RUN orchestrion pin

RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 orchestrion go test -trimpath ./... RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 orchestrion go build -trimpath RUN setcap cap_net_bind_service=+ep /src/fabio

FROM alpine:3.20 RUN apk update && apk add --no-cache ca-certificates COPY --from=build /src/fabio /usr/bin COPY --chown=nobody:nogroup fabio.properties /etc/fabio/fabio.properties USER nobody:nogroup EXPOSE 9998 9999 ENTRYPOINT ["/usr/bin/fabio"] CMD ["-cfg", "/etc/fabio/fabio.properties"]


3. Build a docker image
4. Deploy it locally
5. Register some services in consul
6. Verify the service is present in fabio routing table
7. Observe Traces in DD APM **Additional environment details (Version of Go, Operating System, etc.):**go version go1.23.2 windows/amd64
RomainMuller commented 3 days ago

Hey @djugalov,

Thanks for the detailed report. I am not extremely familiar with consul so I'll need to dig a bit more to make sense of what's supposed to be going on here.

Anyhow - have you had a chance to do the same kind of integration with Consul using "manual instrumentation" with the Datadog tracer library? In essence, there are several reasons that could be at play here:

RomainMuller commented 3 days ago

Also - how is your downstream service routed via Fabio? If it's using TCP routing, I don't think there is support for distributed tracing of these today, and so these wouldn't produce spans...

Generally speaking, it would help us try to reproduce the issue if you could provide us the characteristics of the Fabio routing table.