search

soumith / imagenet-multiGPU.torch

an imagenet example in torch.
BSD 2-Clause "Simplified" License
402 stars 158 forks source link

Changing data path at the time of re-training doesn't make any difference !!! #1

Closed PraveerSINGH closed 9 years ago

PraveerSINGH commented 9 years ago

I started my training with datapath : /home/praveer/raid/imagenet. Ran it till 9 epochs. Then stopped and re-started, this time with the path /home/praveer/ssd/imagenet. For the first few epochs, I had the raid folder mounted so I dint encounter any problem. But as soon as I unmount the raid, it could not trace the imagenet folder. I have re-checked with the datapath (given as -data in the command line) and that seems fine. I initially thought that working with the SSD is equally slow as the HDD but in fact it is the dataloader that has been reading from the HDD instead of switching to the SSD.

szagoruyko commented 9 years ago

I also had this issue when switched from raid to ssd and couldn't understand why it was still slow. Took us some time to understand why.

soumith commented 9 years ago

good point, let me see how i can patch this.

PraveerSINGH commented 9 years ago

thanks @soumith :)

soumith commented 9 years ago

fixed via commit: https://github.com/soumith/imagenet-multiGPU.torch/commit/c2347a5823d3470227d304638af5aea416e99dba