fix(python): release GIL on most operations

delta-io / delta-rs

A native Rust library for Delta Lake, with bindings into Python

https://delta-io.github.io/delta-rs/

Apache License 2.0

1.98k stars 365 forks source link

fix(python): release GIL on most operations #2512

Closed adriangb closed 1 month ago

adriangb commented 1 month ago

Fixes #2269

github-actions[bot] commented 1 month ago

ACTION NEEDED

delta-rs follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

adriangb commented 1 month ago

~~@ion-elgreco could you kick off CI please? It might be worth allowing repeat contributor's to run CI automatically.~~

Never mind, it was just stuck.

ion-elgreco commented 1 month ago

@davidhewitt @adriangb was quite busy this week, so hadn't gotten time to look at until now : )

Yeah it's fine to release the GIL where possible, there was never really a need for it before, so I guess that's why it wasn't brought up outside of that other PR which wasn't finished.

@davidhewitt I think a derive allow_threads is definitely useful if the whole function can release the GIL.

I am not sure, what pyo3 async would provide us though

ion-elgreco commented 1 month ago

Thanks! @adriangb