Closed maaft closed 4 years ago
Hi @maaft
For cell at each position (blue node in Figure 1), it contains two operators: they take the same input, but output with different stride. These two operators do not share weight.
https://github.com/TAMU-VITA/FasterSeg/blob/master/search/model_search.py#L121
Yes, I understand that but what is this "weight sharing" you mentioned in your paper than and where does it happen?
Weight sharing means the subnetworks leverage the same suite of supernet's weights for proxy inference, instead of training each subnetwork from scratch during search. It was first proposed in ENAS. You can find more papers specifically discuss weight sharing on Arxiv. Typical NAS methods like DARTS/ENAS/One-shot all leverage weight sharing.
Hope that helps.
Okay, thank you very much for clarifying! You have been very kind and helpful!
In your paper you write:
This gives the impression that cells actually can choose their downsampling-rate (s = [8, 16, 32] ) freely. But this is not reflected in your code, where you initialize each cell at a fixed downsampling-rate: https://github.com/TAMU-VITA/FasterSeg/blob/bb52a004ff83f64d3dd8989104234fdb862d1cc5/search/model_search.py#L153
Furthermore, when cells are placed at fixed downsampling-rates I don't get how weight sharing according to your paper (above quote) between branches can ever take place as the spatial resolutions always differ between different branches.
tl;dr: Can you elaborate on how you can fuse cells from different branches even when they operate always (according to your code) on different spatial resolutions?