How were paths with skip op being compact ?

VITA-Group / FasterSeg

[ICLR 2020] "FasterSeg: Searching for Faster Real-time Semantic Segmentation" by Wuyang Chen, Xinyu Gong, Xianming Liu, Qian Zhang, Yuan Li, Zhangyang Wang

MIT License

525 stars 107 forks source link

How were paths with skip op being compact ? #19

Closed cwhuang888 closed 4 years ago

cwhuang888 commented 4 years ago

Hi,

Thanks for giving the excellent research on extending NAS into multi-path realm.

While reading alphas2ops_path_width @ model_seg.py (https://github.com/TAMU-VITA/FasterSeg/blob/master/search/model_seg.py#L40), i have noticed that you have also conduct compacting the path when skip is the argmax op. Could you kindly explain briefly about the logic behind this method ?

chenwydj commented 4 years ago

Hi @cwhuang888 ! Thank you for your interest!

When deriving the subnetworks, we shrink the skip_connect by assigning the alpha for the skip_connect operator with value -float('inf'), to avoid selecting the skip_connect.

https://github.com/TAMU-VITA/FasterSeg/blob/master/train/model_seg.py#L54