HMM "nan" Training Improvement

[jiyer2016]

I generated multivariate data using a GammaDistribution and am training a Hidden Markov Model on the data.

The shape of my data is as follows - 1000 sequences - each containing 100 timesteps, with each timestep composed of 3 features:

obs_npy.shape
(1000, 100, 3)

model = HiddenMarkovModel.from_samples(GammaDistribution, n_components=2, X=obs_npy, state_names=['S1','S2',], name='My First Model', verbose=True)

The verbose training output reports an improvement of "nan" and ends almost immediately.

[1] Improvement: nan Time (s): 0.0883 Total Training Improvement: nan Total Training Time (s): 0.2555

I got a feeling that there was some problem with my data - but I was surprised that the model did learn the hidden state transition matrix correctly. The matrix below is pretty accurate - that was the basis of

model.dense_transition_matrix()
array([[0.67832765, 0.32167235, 0.        , 0.        ],
       [0.10658385, 0.89341615, 0.        , 0.        ],
       [0.08945198, 0.91054802, 0.        , 0.        ],
       [0.        , 0.        , 0.        , 0.        ]])

I was thinking - perhaps the model did indeed correctly learn the parameters of the multi-variate gamma-distribution. However, when I checked the model definition - the parameters of the GammaDistribution were all NaN.

                        {
                            "class" : "Distribution",
                            "name" : "GammaDistribution",
                            "parameters" : [
                                NaN,
                                NaN
                            ],
                            "frozen" : false
                        },

I was able to see some legitimate training improvement output when I switched from the GammaDistribution to either NormalDistribution or MultivariateGaussianDistribution - see below:

model = HiddenMarkovModel.from_samples(MultivariateGaussianDistribution, n_components=2, X=obs_npy, state_names=['S1','S2'], 
                                       name='My First Model', verbose=True)

[1] Improvement: 50732.88196909049 Time (s): 0.1567 [2] Improvement: 1290.948022010387 Time (s): 0.1571 [3] Improvement: 108.70278321945807 Time (s): 0.1575 [4] Improvement: 32.18435122468509 Time (s): 0.1563 [5] Improvement: 11.923915891849902 Time (s): 0.157 ... ... ...

Why doesn't it work with the GammaDistribution ?

[jmschrei]

Hey, sorry for the late reply. I had a deadline last night I was trying to make.

What is your data and version of pomegranate? I pushed changes to gamma distributions recently, but can't remember if those fixes are in the latest release or the changes I will be pushing in the next week. Maybe try cloning the latest code and building from source?

[jiyer2016]

Sorry for my delayed response as well. I am using version 0.10.0. However, I'll see if the upgrade works. Thanks.

[jmschrei]

Please re-open if you're still having problems.