search

kolofordjango / kolo

See everything happening in your running Django app. All without leaving VSCode
https://kolo.app
512 stars 14 forks source link

How to view a trace for a script or test that does not include Django? #86

Open zyd14 opened 4 months ago

zyd14 commented 4 months ago

Hi there,

This library looks awesome, but I'd like to apply it to some non-webserver use-cases (primarily data pipelines, but probably some web servers in the future as well). Is there a way to visualize a trace generated for a script or test that doesn't include Django? The docs here describe how to enable tracing of a script or test, but do not describe how you might be able to then visualize the trace. Any guidance would be greatly appreciated, the library looks like it could help solve a lot of observability issues.

wilhelmklopp commented 4 months ago

@zyd14 this is a great point – at the moment the best way to view a non-django/non-web trace is probably via the Kolo VSCode extension: https://docs.kolo.app/en/latest/howto/vscode.html

We've thought about the non-webserver use case a tiny bit in the past and have two ideas for the future:

  1. kolo web serve -> a way to get the local kolo web experience without requiring Django
  2. Uploading traces to Kolo Cloud and then inspecting them there

Don't want to promise anything on when either of these will be available, but we'll likely have the second one ready before the first :)

zyd14 commented 4 months ago

@wilhelmklopp Thanks for the quick response! I was able to use the VSCode extension working for now. Appreciate the insight into future possibilities.

A followup question - is it possible to run kolo programmatically? I'd assume so since you have the middleware setup for Django.

My specific use case is that I would like to get traces from another python tool (Dagster) which is run from the command-line like dagster dev -f <my_code>.py or dagster job run -f <my_code>.py. I tried naively running like kolo dagster job run ... but as I expected that didn't work. If I could somehow instrument my code with kolo then I could probably get traces without having to run kolo from the CLI (I think?). I suppose I can take a look at the middleware to see if I could hack some of the internal methods to my use case - do you have any specific tips for where to start?

Thanks again for the cool library! I'll be keeping my eye on it for sure!

LilyFoote commented 4 months ago

Does dagster provide a python -m dagster interface? If so, I think kolo run python -m dagster might work for you.