search

GoogleCloudPlatform / dataflux-pytorch

The Dataflux Accelerated Dataloader for PyTorch with GCS is an effort to improve ML-training efficiency when using data stored in GCS for training datasets. Using the Dataflux Accelerated Dataloader for training is up to 3X faster when the dataset consists of many small files (e.g., 100 - 500 KB).
Apache License 2.0
26 stars 4 forks source link

update to latest dataflux client #101

Closed jdnurme closed 1 month ago

jdnurme commented 1 month ago

Attempting to fix broken unit tests by updating client library

It's a good idea to open an issue first for discussion.