single-cell-data / SOMA

A flexible and extensible API for annotated 2D matrix data stored in multiple underlying formats.
MIT License
69 stars 9 forks source link

Prevent deadlocks in EagerIterators by making prefetch optional. #185

Closed thetorpedodog closed 6 months ago

thetorpedodog commented 6 months ago

Previously, if you provided a thread pool that was too small and an EagerIterator could not create a new preloading thread, the iterator would deadlock, since it would wait for the new thread to be created forever and not try to just do the work itself. This change instead uses preloading as an optional optimization, and if the preload has not yet been completed, computes the next value itself.