Load image paths and labels into DataFrame
The dataset images were loaded into NumPy arrays, and saved as .npy files. This allows them to be loaded up really fast, preventing the data loading process from bottlenecking the GPU training.
When loading the images from JPEGs, the CPU is at max usage and the GPU usage dips occasionally between batches.
When loading the images from .npy files, the CPU usage is less than max and the GPU usage is more consistent. So it helps :)
Pipe-Runner
commented
4 years ago
Load image paths and labels into DataFrame The dataset images were loaded into NumPy arrays, and saved as .npy files. This allows them to be loaded up really fast, preventing the data loading process from bottlenecking the GPU training.
When loading the images from JPEGs, the CPU is at max usage and the GPU usage dips occasionally between batches.
When loading the images from .npy files, the CPU usage is less than max and the GPU usage is more consistent. So it helps :)