search

htjb / margarine

Code to replicate posterior probability distributions with bijectors/KDEs and perform marginal KL/bayesian dimensionality calculations.
MIT License
13 stars 8 forks source link

Temperamental tests #37

Open htjb opened 1 year ago

htjb commented 1 year ago

Some of the tests occasionally fail because of poor training.

htjb commented 6 months ago

Okay, in the tests we calculate the accuracy of the MAFs and I think KDEs too by running a series of two sample KS tests between samples drawn from the DE and the original samples used to train. Occasionally one of these KS tests fails because the training is poor (training varies with the random seed since the weights are initialised differently).

A simpler thing to do would be a KS test between the predicted distributions of log-probability vs the log-probability of the original samples. Might be more robust. Also I think the tests currently work with a NS run but should use something from the scipy.stats package.

Also appears there is a bug with tensorflow too after their recent updates (identified in #59 I think).

htjb commented 6 months ago

Would be good to remove the pytorch dependency from the tests as well. Seems to take a long time to install and not really needed.

htjb commented 6 months ago

and test a wider range of python versions.