search

JiaminRen / RandWireNN

Pytorch Implementation of: "Exploring Randomly Wired Neural Networks for Image Recognition"
274 stars 42 forks source link

Question on additive aggregation weight initialization #8

Closed seungwonpark closed 5 years ago

seungwonpark commented 5 years ago

Hi, I have a question about weight initialization of node's additive aggregation here.

At model.py#L41, weights are initialized with torch.ones, not torch.zeros. This will make actual weights to be initialized with 0.7311, not 0.5000. Was it intended? (Though it won't affect much on training, I'm just curious about it.)

JiaminRen commented 5 years ago

I don't think it matters. A single parameter should be easy to optimize.

seungwonpark commented 5 years ago

Thanks for the comment.