JuliaParallel / Dagger.jl

A framework for out-of-core and parallel execution
Other
610 stars 66 forks source link

Add render/show_logs to chrome_trace #540

Closed m-fila closed 1 week ago

m-fila commented 1 week ago

Chrome tracing is a fairly popular tool for working with traces and process timelines. The traces can be not only displayed but also analyzed and queried with SQL

The Dagger logs can be transformed to a chrome-trace schema json format. The resulting json file can be loaded to https://ui.perfetto.dev/ or legacy about:tracing in a chrome-based browser

Dagger.enable_logging!(tasknames=true)
...
logs = Dagger.fetch_logs!()
open("chr_trace.json", "w") do io
    Dagger.show_logs(io, logs, :chrome_trace)
end

This can be considered an alternative to the in-house :plots_gantt


Example output (json file):

perfetto

For comparison, the same logs printed with :plots_gantt

gantt


Optional dependency on JSON3 is added, but it's used only to get a json-string from a Dict so it should be possible to easily replace it with any other JSON library in case you have a preference

For now only :compute events are used (same as in :plots_gantt) but it should be possible to add support for other types of evetns too

jpsamaroo commented 1 week ago

Sorry for the delay - this is really cool! I'll review this soon, it looks already really good :heart:

jpsamaroo commented 1 week ago

Thanks so much @m-fila !