Allow running demo code in listing-only mode

GoogleCloudPlatform / dataflux-pytorch

The Dataflux Accelerated Dataloader for PyTorch with GCS is an effort to improve ML-training efficiency when using data stored in GCS for training datasets. Using the Dataflux Accelerated Dataloader for training is up to 3X faster when the dataset consists of many small files (e.g., 100 - 500 KB).

Apache License 2.0

26 stars 4 forks source link

Allow running demo code in listing-only mode #95

Closed abhibyreddi closed 1 month ago

abhibyreddi commented 1 month ago

Training in skipped when --listing_only is passed to train.py and exits right after initializing the map style datasets.

[X] Tests pass; ran locally and verified listing time is output when this flag is set
[X] Appropriate changes to documentation are included in the PR