mattjj / pyhsmm

MIT License
548 stars 173 forks source link

The code is not numerical stable, it throws runtime warnings from time to time #85

Open kunrenzhilu opened 6 years ago

kunrenzhilu commented 6 years ago

I test my model with the following setting and it throws an runtimewarning every time. I tested the model with the same setting in py2. Sometimes it's safe but sometimes the same error throws. The data is just some trajectories with state labels. range from 0 to 64*64-1. And total length of each observation is 8640.

Any hint on why it's not stable in py2 and not event work in py3?

/usr/local/lib/python3.5/dist-packages/scipy/stats/_distn_infrastructure.py:899: RuntimeWarning: divide by zero encountered in log
  return log(self._sf(x, *args))````

```python
Nmax = 24
obs_hypparams = {'K':64*64, 'alpha_0':64*64, 'alphav_0':np.ones((64*64,))} # K is the cardinality of the states, and the alpha is the prior trial.
dur_hypparams = {'alpha_0':8640, 'beta_0':2} # in beta interval, the event happens for alpha time

obs_distns = [distributions.Categorical(**obs_hypparams) for state in range(Nmax)]
dur_distns = [distributions.PoissonDuration(**dur_hypparams) for state in range(Nmax)]

categorical_post = pyhsmm.models.WeakLimitHDPHSMM(
        alpha_a_0=1., alpha_b_0=1./4,
        gamma_a_0=6., gamma_b_0=1./4,
        init_state_concentration=6.,
        obs_distns=obs_distns,
        dur_distns=dur_distns)

for i in range(10):
    categorical_post.add_data(hsmm_trajs[i], trunc=60)

from pyhsmm.util.text import progprint_xrange
import copy
models = []
for idx in progprint_xrange(20):
    categorical_post.resample_model(10)
    if (idx+1) % 10 == 0:
        models.append(copy.deepcopy(categorical_post))
kunrenzhilu commented 6 years ago

I found the way to parallel already.

bacalfa commented 6 years ago

I'm able to run the examples on a Mac, but the HSMM examples crash on Windows (after I built the code with the modifications in this pull request https://github.com/mattjj/pyhsmm/pull/86).

bacalfa commented 6 years ago

Update: while it crashes after building and installing using Visual Studio's cl.exe, it works fine if built with MinGW32-x64. I had to make other modifications described below.