[feat] More flexible + implement a filesystem based dataloader variant

Photoroom / datago

A golang-based data loader which can be used from Python. Processing data per sample at GB/s speeds, covering various use cases eventually.

MIT License

82 stars 0 forks source link

[feat] More flexible + implement a filesystem based dataloader variant #19

Closed blefaudeux closed 3 weeks ago

blefaudeux commented 1 month ago

WIP [x] Refactor to enable multiple data pointers generators & backends [x] Implement a file system based option [ ] Benchmark against pytorch dataloader on ImageNet

blefaudeux commented 3 weeks ago

no time to work on this at the moment, but would be a good demo.. Will update and revive in due time