Threaded data loading

[dasguptar]

Looking at the addjob call here it seems that each thread loads a batch, which is then sent to the main thread to be passed through the model. However, this is happening in a multi-GPU setting, with data and model parallelism. In a single GPU setting, if I want to have a thread that loads a batch and augments the data, while another thread passes the batch through the model, how do I go about implementing such a scenario?

[soumith]

In a single GPU setting, if I want to have a thread that loads a batch and augments the data, while another thread passes the batch through the model, how do I go about implementing such a scenario?

Currently if you run it in the single-GPU case then:

• the threadpool loads a mini-batch + augments it with random crop etc.
• the main thread passes the batch through the model. The main thread is not blocked on the thread-pool loading batches in the background.

[dasguptar]

So basically, I have to make a grand total of 0 changes? This was incredibly dumb of me to ask in that case. Thanks a ton.