pola-rs / polars

Dataframes powered by a multithreaded, vectorized query engine, written in Rust
https://docs.pola.rs
Other
29.96k stars 1.93k forks source link

Clarification in documentation on whether polars-rust can be used directly in async or not #4432

Closed SimonSchneider closed 2 years ago

SimonSchneider commented 2 years ago

I'm unsure myself but think it would be beneficial to include a small note in the documentation somewhere whether polars can be used directly in async contexts in rust or if it should be wrapped in tokio::task::spawn_blocking (when using tokio).

I think it could be beneficial to include something similar as arrow2 in the readme or rust docs if it is possible to use in an async context, or to mention that it is not suitable to be used in an async context. https://github.com/jorgecarleitao/arrow2/blob/9066298b531054ad7c0253827dd421797bf5684c/README.md

Fully-decoupled IO between CPU-bounded and IO-bounded tasks, allowing this crate to both be used in async contexts without blocking and leverage parallelism

ritchie46 commented 2 years ago

Arrow2 has both sync and an async API, so I understand that note in the docs.

We don't have an async API. Could you make a PR and add a section where you think this should be mentioned?

SimonSchneider commented 2 years ago

That's true, perhaps this is rather obvious, I was not aware that arrow2 had an async API. perhaps this is not needed.

Thinking about it now it's rather obvious that it should be wrapped in spawn_blocking. Do you think there's a point in mentioning async in the rust documentation at all or will people understand this? Just including the keyword async in the docs with a small note could potentially help newcomers.

ritchie46 commented 2 years ago

Could you make a PR? I have no strong opinion here.