guswynn
commented
8 months ago tokio-console maintains what "child" futures are currently being polled in a task and a history of what has executed. Another way to think about this would be a streaming view of OpenTelemetry traces.
this is partially covered by the unstable tasktrace feature of tokio, but the history idea is interesting!
@ParkMyCar I think this is my main list of things that would help us here:
- stabilization of the core
tokio-consoleprotocol, so we can build mz without--cfg=tokio_unstable- This requires confidence that we can bound the memory usage of the feature, and that it has consistent cpu usage. even better would be being about to turn it on dynamically (this may already be possible but I havent tested it)
- This also includes adding async resource support for mpsc channels (and other types in
tokio::sync
- Support for tagging tasks with arbitrary
tracingfields. This would looks nice intokio-console, even better if you can search by it- Similarly, more clarity in the
tokio-consoleui on which tasks are referring to which channels, and how they are connected
- Similarly, more clarity in the
- similarly, stabilization of the
tasktracefeature oftokio- The unstable entrypoint that exists: https://docs.rs/tokio/latest/tokio/runtime/struct.Handle.html#method.dump is quite nice, we can expose it over http easily, but perhaps even nicer would be some kind of
tracingintegration
- The unstable entrypoint that exists: https://docs.rs/tokio/latest/tokio/runtime/struct.Handle.html#method.dump is quite nice, we can expose it over http easily, but perhaps even nicer would be some kind of
- help managing issues and complexities with tracing
- the main ones here are: panics in the tracing-otel layer when behind a
reload::Layer, and issues around sharing registries among multiple subscribers (for things like theEXPLAINimpl)
- the main ones here are: panics in the tracing-otel layer when behind a
- help bounding the amount of otel tracing we produce
- its possible good documentation here about how to best manage spans across many services/channels/tasks moves this needle quite a bit. even just deciding on levels is difficult
- stabilization of the tokio runtime metrics apis, and better examples on how to expose them to prometheus
Ill add stuff if I think there is more, but I think this is my main list!
ParkMyCar
This issue is intended to track any work related to improving debugging async Rust, and serve as a place to centralize all of the improvements we would like to see.
Improvements
tokio-consolemaintains what "child" futures are currently being polled in a task and a history of what has executed. Another way to think about this would be a streaming view of OpenTelemetry traces.tokio-consoleassociates async resources, e.g. locks and channels, with the tasks current awaiting them.