Templnet

[Ithanil]

The former FFNN implementation (now referred to as PolyNet) is retired from development for the moment, but remains and will be kept functional and possibly overhauled in the future. Added new FFNN implementation named TemplNet:

• TemplNet requires to specify network configuration (sizes, activation functions) at compile time and thereby exploits C++ template programming to "bake" the network structure and all methods in code. This promises more performance, allows for many static sanity checks and the compile time requirement can be easily handled in some use cases.
• As opposed to the current PolyNet, TemplNet computes all first order derivatives (w.r.t. inputs or weights) by a reverse differentiation scheme, as used in the well-known backprop learning. It was found that the old forward-accumulation scheme was still the most favorable to compute the second order input derivatives and therefore remained in that case.
• TemplNet is tested in unit tests and validated against PolyNet. However, the current implementation should be considered just as a first iteration and somewhat experimental in concept.
• Depending on the network and derivative configuration, TemplNet was found to achieve between 2x and 100x speedup versus the current PolyNet.

[coveralls]

Pull Request Test Coverage Report for Build 314

• 497 of 537 (92.55%) changed or added relevant lines in 14 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+2.07%) to 82.575%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
include/qnets/poly/FeedForwardNeuralNetwork.hpp	14	15	93.33%
include/qnets/templ/TemplNet.hpp	116	117	99.15%
src/poly/FeedForwardNeuralNetwork.cpp	35	49	71.43%
include/qnets/templ/TemplLayer.hpp	160	184	86.96%
<!--	Total:	497	537	92.55%	-->

Totals
Change from base Build 277:	2.07%
Covered Lines:	3265
Relevant Lines:	3954

---

💛 - Coveralls