search

google / Xee

An Xarray extension for Google Earth Engine
https://xee.rtfd.io
Apache License 2.0
253 stars 29 forks source link

Could we increase CPU-bound performance by manually managing the GIL? #169

Open alxmrs opened 2 months ago

alxmrs commented 2 months ago

I’ve done some research on this for Cubed: https://github.com/cubed-dev/cubed/issues/497

I’m curious that if it would be worth it to add a Cython build step to Xee. From SO and docs that I’ve read, there may be more CPU performance we can squeeze out if we manually manage the GIL when interacting with concurrent ThreadPools.

I suspect @shoyer may know if this is an issue, but measuring is always better than guessing.

alxmrs commented 2 months ago

This relates to https://github.com/google/Xee/issues/29.

naschmitz commented 1 month ago

Good idea!

We're looking into EE backend improvements first. We'll queue this work up behind those improvements.