This adds the ability for networks to use skip/residual connections. Most of the changes are based on their implementation in the LSP-Lab repository.
This is done by adding a new Layer object, skipConnection, for adding residual connections to networks. The nature of residual connections means that the basic forward_pass function also had to be changed to accommodate them.
Previous test models and unit tests still work with the forward pass changes and a couple of new tests check that the residual connections can be added and work at a basic level.
Some documentation was added for the new layer as well, but I consider it a draft at best and would welcome edits to it.
This adds the ability for networks to use skip/residual connections. Most of the changes are based on their implementation in the LSP-Lab repository.
This is done by adding a new Layer object,
skipConnection, for adding residual connections to networks. The nature of residual connections means that the basicforward_passfunction also had to be changed to accommodate them.Previous test models and unit tests still work with the forward pass changes and a couple of new tests check that the residual connections can be added and work at a basic level.
Some documentation was added for the new layer as well, but I consider it a draft at best and would welcome edits to it.