Lying somewhere between a bug and a feature enhancement, the current iteration of the nn.Module.children assumes that the layers are in a sequence (like a significantly deep convolutional network).
It does not however, consider that the children it encounters are all siblings, rather than always being parent-child relationships.
Given how nn.Module represents its layer relationships, crafting a solution for many cases for this could be very tricky.
Some architectures that may be negatively impacted by this:
U-Net
DenseNets (mentioned in the presentation)
Any network that trains on multiple outputs
Depending on how its modeled, these could all be considered sequential children or a nested output.
Lying somewhere between a bug and a feature enhancement, the current iteration of the
nn.Module.childrenassumes that the layers are in a sequence (like a significantly deep convolutional network).It does not however, consider that the children it encounters are all siblings, rather than always being parent-child relationships.
nn.Modulerepresents its layer relationships, crafting a solution for many cases for this could be very tricky.Some architectures that may be negatively impacted by this: