melodyguan / enas

TensorFlow Code for paper "Efficient Neural Architecture Search via Parameter Sharing"
https://arxiv.org/abs/1802.03268
Apache License 2.0
1.58k stars 390 forks source link

Could you clarify why the `num_outs` is counted with `tf.where(tf.equal(used, 0))` ? #75

Open hubert0527 opened 6 years ago

hubert0527 commented 6 years ago

Hi,

Thanks for the great work and sharing the implementation to the community! I recently found that this part of codes looks kind of weird.

During micro-search for CIFAR-10, you first count the number of precious layers which is NOT picked during current _enas_layer construction: https://github.com/melodyguan/enas/blob/d1a90ac915301198f2a30ce136e9040e6c4235ff/src/cifar10/micro_child.py#L660

Then count how many layers are NOT picked, and save the value to num_outs: https://github.com/melodyguan/enas/blob/d1a90ac915301198f2a30ce136e9040e6c4235ff/src/cifar10/micro_child.py#L663

The naming of num_outs itself looks kind of weird and it is then used to reshape the output tensor, which looks logically weird. https://github.com/melodyguan/enas/blob/d1a90ac915301198f2a30ce136e9040e6c4235ff/src/cifar10/micro_child.py#L674

Could you clarify why num_outs uses the number of previous layers which are NOT picked? Sincerely sorry if I misunderstand anything.

Thanks!