About `dropout step` in the paper

luzai commented 5 years ago

Thank you very much for sharing the whole implementation! I am curious about dropout step in this figure, may I ask some questions?

As shown in the code here, I guess that dropout on indicator is applied during the warmup steps (6255 steps), the keep probability (rather than drop probability) is 0.6; after 6255 steps, the keep probability is 100, and the model will choose 3x3, 5x5, exp=3, exp=6 by learned threshold. May I ask whether my interpretation correct?
May I ask why the runtime term is larger in the dropout steps? And why the runtime decrease rapidly and precisely to 79 ms? What whould the curve be like if hyperparameter lambda=0.02 is setted to other values?

annamz6526-1 commented 5 years ago

Hi @luzai. I think your interpretation for the dropout is correct. But I couldn't figure out why they set the keep probability to 100, which means to scale the Indicator value 100 times smaller.

luzai commented 5 years ago

It seems that keep_prob must be in (0,1], as shown in here. But the code run successfully without ValueError.

QueeneTam commented 5 years ago

It seems that keep_prob must be in (0,1], as shown in here. But the code run successfully without ValueError.

It seems that keep_prob must be in (0,1], as shown in here. But the code run successfully without ValueError. Hello, there are some exceptions when I run the code, may you give me some advise about this dropout issue? Mant thanks! Question link: https://github.com/dstamoulis/single-path-nas/issues/11#issuecomment-549108585

qianhuiliu commented 2 years ago

I meet the same question. Have you figured it out?

enyac-group / single-path-nas

About `dropout step` in the paper #6