search

D-X-Y / AutoDL-Projects

Automated deep learning algorithms implemented in PyTorch.
MIT License
1.56k stars 281 forks source link

feature maps weighted sum by gumbel softmax #61

Closed chunhuizng closed 4 years ago

chunhuizng commented 4 years ago

Hello, thanks for your great work! I want to ask you a question: In lib/models/cell_searchs/search_cells.py, line58, the weighted-sum in Gumbel-softmax is calculated as follows: weigsum = sum( weights[_ie] * edge(nodes[j]) if _ie == argmaxs else weights[_ie] for _ie, edge in enumerate(self.edges[node_str]) ). weights[_ie] * edge(nodes[j]) is tensor but weights[_ie] is scalar. weights[_ie] does not multiply with a tensor like edge(nodes[j]). So weights[_ie] * edge(nodes[j]) and weights[_ie] do not have the same dimension.

However, codeweigsum = sum( weights[_ie] * edge(nodes[j]) if _ie == argmaxs else weights[_ie] for _ie, edge in enumerate(self.edges[node_str]) ) is all right when running. Why weights[_ie] * edge(nodes[j]) and weights[_ie] can be added together even if both dimensions(data.shape) are different ?

Looking forward to your reply thanks.

D-X-Y commented 4 years ago

First of all, thanks for using my repo. In PyTorch, the operations on tensor can support broadcast for */+, if you add/multiply a scalar with a tensor, it is ok.