BiomedSciAI / fuse-med-ml

A python framework accelerating ML based discovery in the medical field by encouraging code reuse. Batteries included :)
Apache License 2.0
134 stars 34 forks source link

shared memory utility helping to speedup reading from files significantly if enough RAM is available #347

Closed YoelShoshan closed 6 months ago

YoelShoshan commented 6 months ago

shared memory utility helping to speedup reading from files significantly if enough RAM is available

seen at least x10 speedup.

on a machine that is already bottlenecked on disk seen over x100 speedup