search

UCBerkeleySETI / hyperseti

A SETI / technosignature search code to find intelligent life beyond Earth
https://hyperseti.readthedocs.io
11 stars 4 forks source link

Asyncio dev #81

Closed telegraphic closed 1 year ago

telegraphic commented 1 year ago

Test using asyncio within pipeline.py. Not sure what h5py and cupy do when they are processing (are they blocking?)

telegraphic commented 1 year ago

Indeed, no speed up.

(Aim is to allow reading from disk for next gulp while processing.)

telegraphic commented 1 year ago

Some background: https://github.com/h5py/h5py/issues/837

In short, h5py does not (yet) support async I/O, and HDF5 has only recently added support. So no performance gains possible without actual pipelining a la bifrost.