search

rajveerb / lotus

Lotus: Characterization of Machine Learning Preprocessing Pipelines via Framework and Hardware Profiling
Other
1 stars 1 forks source link

Code to use dataloader more than the number of GPUs #14

Closed rajveerb closed 1 year ago

rajveerb commented 1 year ago

In the current code in main branch, the program uses all the GPUs available on the node. The -j argument only controls the number of data loader workers created and does not specify number of GPUs to be used.

Under this issue, I am adding a patch to separate the logic.

The user will explicitly specify the number of GPUs, using --gpus to be used.