search

ml-explore / mlx-data

Efficient framework-agnostic data loading
MIT License
362 stars 40 forks source link

Would an async api have performance gain? #7

Open npuichigo opened 11 months ago

npuichigo commented 11 months ago

Fantastic job! But I'd like to ask if there would be some another gains by designing an async implementation and bind them to python with __anext__ and __aiter__, since data loading is highly related to IO.

angeloskath commented 11 months ago

Thanks! I don't think asynchronous iteration would be helpful in our particular case.

Background fetching of the next sample or batch is achieved using the prefetch method. This leaves the python thread to process the batch while in the background other threads are fetching the next samples. If you have a particular use case that we might have missed which would benefit from python's asyncio then we 'd love to reevaluate.

npuichigo commented 11 months ago

If we have a slow IO like remote cloud storage like S3, maybe it would block the prefetch thread pool?