libffcv / ffcv

FFCV: Fast Forward Computer Vision (and other ML workloads!)
https://ffcv.io
Apache License 2.0
2.84k stars 178 forks source link

S3/remote storage of beton files #309

Open igor-krawczuk opened 1 year ago

igor-krawczuk commented 1 year ago

Hi, I wanted to ask, is accessing .beton files that are stored in a S3 bucket or some other remote filesystem supported/considered in the design of the library (i.e., does it handle caching on the ffcv side)?

Thanks for the amazing library in any case, it has helped some of my experiments greatly