dynatrace-oss / OneAgent-SDK-Python-AutoInstrumentation

autodynatrace, a python library that implements automatic instrumentation using the OneAgent SDK for Python
Other
62 stars 28 forks source link

Code Level Tracing #64

Closed jmeisele closed 2 years ago

jmeisele commented 2 years ago

I have instrumented a FastAPI application running via gunicorn by using AUTOWRAPT_BOOTSTRAP=autodynatrace & AUTODYNATRACE_FORKABLE=true. I can see the process level in Dynatrace but nothing deeper as far as code level tracing or ingested traces. Is there another step I am missing in my application? Do I need to add the decorator @autodynatrace.trace() in each of my routes? I don't see any distributed traces in Dynatrace for this application, the W3C traceID is not getting captured for any of the routes which seems suspicious to me. Love the library by the way ❤️ 🚀

image

dlopes7 commented 2 years ago

Hi @jmeisele

No, you should not need to instrument your routes, something is missing here.

Could you please share:

Also, what do you get when you hit that "View distributed traces" button? No traces are shown?

jmeisele commented 2 years ago

Hello @dlopes7 👋🏻 ☕ Thanks for the fast response!

fastapi version 0.75.2

Launch command gunicorn -k uvicorn.workers.UvicornWorker --bind=0.0.0.0:3000 --workers=1 app.main:app --preload

When I click the distributed traces icon I get:

image

But if I click one of the traces it can't find the trace.

image

I cross referenced what was different in the URL from this trace attempt vs another service in GoLang and I noticed the traceID is not there.

image

image

dlopes7 commented 2 years ago

This is interesting, as this is supposed to be a good old purepath (not a distributed trace per se, so W3C trace ID is not at play)

I am suspecting this might be a cluster side issue, as it seems your requests are being instrumented correctly.

Let me try to reproduce this in a couple of different environments here, I might need help from our cluster side R&D

jmeisele commented 2 years ago

Much appreciated, I was mostly looking for the differences, so W3C trace ID could be totally unrelated 😅

dlopes7 commented 2 years ago

Just an for the records

App: https://github.com/dlopes7/autodynatrace-gunicorn-test

Dynatrace 1.250

No 404 response, managed to get to the trace details

URL: /#trace;gf=all;traceId=20858f3f5d95cee8445e889f33840973;timeframe=custom1660919815409to1660919936411;callURI=64185111Z1X6X0X512348X20220819142652Y0Y0.session

And W3C trace ID is indeed present W3C trace ID​ 20858f3f5d95cee8445e889f33840973

jmeisele commented 2 years ago

I do not have these 2 lines in my Dockerfile, do I need to add?

dlopes7 commented 2 years ago

No, that is just a way of installing the OneAgent "inside" the container without needing to do anything outside, it also allows me to test different environments without changing anything other than the image.

But this can be done in several different ways (installing on the host, installing with the kubernetes operator, etc)

dlopes7 commented 2 years ago

Dynatrace 1.249

No 404 response, managed to get to the trace details

URL: #trace;gf=all;traceId=1031789920d06fa8e8eba2f3ef907d4a;timeframe=custom1660920540932to1660920661935;callURI=13428147Z1X9X0X32024X20220819141805Y0Y0.session

Trace ID: 1031789920d06fa8e8eba2f3ef907d4a

@jmeisele what is your Dynatrace version?

jmeisele commented 2 years ago

Looks like we're on 1.247

image

dlopes7 commented 2 years ago

Ok, I don't have a 1.247 environment to test atm, will have to come back to this on monday

jmeisele commented 2 years ago

@dlopes7 we had an upgrade over the weekend to version 1.248 and now I can see code level tracing just fine.

Closing this issue out 🤝

image

image