Apart from refactoring GDLearning, this also takes out use_ivs or with_ivs.
It extends Weights class with a normalize method that takes care of normalizing the weights internally. I found that using assign or assign_log (as was done previously in GDLearning) for normalization yielded incorrect results, resulting in an SPN unable to learn.
The new GDLearning isolates the loss functions for generative and discriminative learning, so that they can still be constructed through the public API (previously this happened strictly internally).
The new GDLearning also isolates the post-gradient update ops, such as normalizing weights and clipping the scale parameter of GaussianLeaf nodes. Now users can construct their own update ops and are no longer restricted to the internal construction of the minimization op as produced by a tf.train.Optimizer.
The new GDLearning does not support non-log inference. First of all, it seems redundant to do so (it is constructed internally anyway). Second, non-log inference causes underflow. Thirdly, the already existing implementation was not correctly constructing the loss functions in case of non-log inference.
Added the options for L1 and L2 regularization, which will have to be assessed empirically.
GDLearning, this also takes outuse_ivsorwith_ivs.Weightsclass with anormalizemethod that takes care of normalizing the weights internally. I found that usingassignorassign_log(as was done previously inGDLearning) for normalization yielded incorrect results, resulting in an SPN unable to learn.GDLearningisolates the loss functions for generative and discriminative learning, so that they can still be constructed through the public API (previously this happened strictly internally).GDLearningalso isolates the post-gradient update ops, such as normalizing weights and clipping the scale parameter ofGaussianLeafnodes. Now users can construct their own update ops and are no longer restricted to the internal construction of the minimization op as produced by atf.train.Optimizer.GDLearningdoes not support non-log inference. First of all, it seems redundant to do so (it is constructed internally anyway). Second, non-log inference causes underflow. Thirdly, the already existing implementation was not correctly constructing the loss functions in case of non-log inference.