search

JuliaParallel / Dagger.jl

A framework for out-of-core and parallel execution
Other
638 stars 67 forks source link

TimespanLogging: Provide log serialization helper #381

Open jpsamaroo opened 1 year ago

jpsamaroo commented 1 year ago

It's super common to want to export logs to a file to be later imported for analysis. We could provide an interface to export to any Tables.jl-compatible sink to make this easy.

An open question is how to serialize streams of a MultiEventLog, which tend to have different formats. These also often contain data like Dict or NamedTuple which don't serialize nicely as column values in CSV; maybe we should also have a JSON-based format? If we emit JSON, we could also move most of the sanitization code from DaggerWebDash over to TimespanLogging.

findmyway commented 1 year ago

Hi @jpsamaroo ,

I'm working on https://github.com/oolong-dev/OpenTelemetry.jl/pull/74 recently. I believe most logging related issues listed here can be addressed gracefully. Maybe we can have a discussion on it in the next Dagger.jl user meeting?

jpsamaroo commented 1 year ago

@findmyway sounds great to me!