Mixture weights are NaNs for MoGs

[jonathanoesterle]

Hi,

When I use a Mixture of Gaussians with at least 2 components and I set svi=True in SNPE, then the mixture weights a sometimes become NaNs. I assume this happens in the softmax-function, but I'm not sure. The error is reproducible and does not happen (with the same data) if I have only 1 component. It also does not happen when I set svi=False.

Best, Jonathan

Here, the full error message.

~/berens/python_libs/delfi/delfi/inference/SNPE.py in run(self, n_train, n_rounds, epochs, minibatch, round_cl, stop_on_nan, proposal, text_verbose, monitor, load_trn_data, save_trn_data, append_trn_data, init_trn_data_folder, **kwargs)
    247             if text_verbose: print('Done!')
    248             try:
--> 249                 posteriors.append(self.predict(self.obs))
    250             except np.linalg.LinAlgError:
    251                 posteriors.append(None)

~/berens/python_libs/delfi/delfi/inference/BaseInference.py in predict(self, x, deterministic)
    350         """
    351         x_zt = (x - self.stats_mean) / self.stats_std
--> 352         posterior = self.network.get_mog(x_zt, deterministic=deterministic)
    353         return posterior.ztrans_inv(self.params_mean, self.params_std)
    354 

~/berens/python_libs/delfi/delfi/neuralnet/NeuralNet.py in get_mog(self, stats, deterministic)
    300         Us = [comps['U' + str(i)][0] for i in range(self.n_components)]
    301 
--> 302         return dd.MoG(a=a, ms=ms, Us=Us, seed=self.gen_newseed())
    303 
    304     def reseed(self, seed):

~/berens/python_libs/delfi/delfi/distribution/mixture/GaussianMixture.py in __init__(self, a, ms, Ps, Us, Ss, xs, seed)
     50                 ndim=np.asarray(
     51                     ms[0]).ndim,
---> 52                 seed=seed)
     53 
     54             if Ps is not None:

~/berens/python_libs/delfi/delfi/distribution/mixture/BaseMixture.py in __init__(self, a, ncomp, ndim, seed)
     36             self.rng = np.random.RandomState()
     37 
---> 38         self.discrete_sample = Discrete(p=self.a, seed=self.gen_newseed())
     39 
     40     @abc.abstractmethod

~/berens/python_libs/delfi/delfi/distribution/Discrete.py in __init__(self, p, seed)
     19         p = np.asarray(p)
     20         assert p.ndim == 1, 'p must be a 1-d array'
---> 21         assert np.isclose(np.sum(p), 1), 'p must sum to 1 but sum is ' + str(np.sum(p)) + ' with elements ' + str(p)
     22         self.p = p
     23 

AssertionError: p must sum to 1 but sum is nan with elements [nan nan nan]

[dgreenberg]

Hi Jonathan, could you please provide code to reproduce this error?

[jonathanoesterle]

I changed a lot in my code and now I can't get this error anymore, even when I try. As soon as it occurs again, I will post it here. Sorry, I can't help at the moment.

[dgreenberg]

Ok, I'll close the issue for now. Feel free to reopen it if the error pops up again.