Arbol-Project / dcdf

dClimate Data Format
2 stars 0 forks source link

Investigate async runtimes for Python layer #26

Open chrisrossi opened 1 year ago

chrisrossi commented 1 year ago

The Rust code is written using async i/o in a way that is agnostic to the runtime being used. Choosing, setting up, and invoking a runtime is the responsibility of the caller. Since the Python layer wraps the Rust async API with a synchronous one, the Python layer chooses and invokes a runtime. The one used currently is just whatever you get when you invoke "tokio::runtime::Runtime::new()".

I've noticed when running encoding jobs in Python that it only seems to be using one core on my four core machine, so it's probably worth investigating and learning about the runtimes available from tokio and/or other async libraries to see if there is one that is better suited.