For DARTS search space-with Imagenet-1k dataset, the time reported in the paper (4 hours) is not even close when replicated.

VITA-Group / TENAS

[ICLR 2021] "Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective" by Wuyang Chen, Xinyu Gong, Zhangyang Wang

MIT License

167 stars 31 forks source link

For DARTS search space-with Imagenet-1k dataset, the time reported in the paper (4 hours) is not even close when replicated. #20

Closed oshindutta closed 2 years ago

oshindutta commented 2 years ago

For DARTS search space-with Imagenet-1k dataset, the time reported in the paper (4 hours) is not even close when replicated on Tesla v100 (it should be lower than 4 hours as v100 is much faster than 1080 Ti).

It takes around 10 hours with the batch_size 24 (as mentioned in the code).

What do you think might be the issue?

chenwydj commented 2 years ago

Hi @oshindutta,

Thanks for your interest in our work!

The "four GPU hours" is verified multiple times, so it is valid.

Possible reasons might be on other hardware sides: the CPU load, the I/O speed of your hard disk, etc.