Open carljohnsen opened 1 year ago
Rather that the following structure (disk > ram > blockwise GPU), we should make it load the data asynchronously through a python generator, which only loads blocks on demand when transferring to the GPU.
generator
Maybe even look at if there exists an asynchronous version of mmap
mmap
Rather that the following structure (disk > ram > blockwise GPU), we should make it load the data asynchronously through a python
generator
, which only loads blocks on demand when transferring to the GPU.