I guess in this line, linear_layers should be layers_xyz?
In a similar realm, I guess in this line you don't want a skip connection in the last layer? Then i != num_layers - 1 should be i != num_layers - 2, as the for-loop is in range(num_layers - 1) already? i will never be num_layers - 1, so it could happen that you have a skip-connection in the last layer of layers_xyz, e.g. if skip_connect_every=4 and num_layers=10. This should then throw an error in the forward call in this line. As there would be a dimension mismatch.
I guess in this line,
linear_layersshould belayers_xyz?In a similar realm, I guess in this line you don't want a skip connection in the last layer? Then
i != num_layers - 1should bei != num_layers - 2, as the for-loop is inrange(num_layers - 1)already?iwill never benum_layers - 1, so it could happen that you have a skip-connection in the last layer oflayers_xyz, e.g. ifskip_connect_every=4andnum_layers=10. This should then throw an error in theforwardcall in this line. As there would be a dimension mismatch.