Issue about mcmc_samples

[liliphhl]

when i use the following code, it will propose an exception: normal_lpdf: Scale parameter is 0, but must be >0!

Python

m = Prophet(mcmc_samples=500) forecast = m.fit(df).predict(future)

[bletham]

When I see this message it is followed up by "If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine, but if this warning occurs often then your model may be either severely ill-conditioned or misspecified."

If you are only seeing this at the beginning of sampling, then it isn't an issue, it's just the MCMC sampler warming up. Is that the case?

[bletham]

I'm guessing this was resolved but feel free to reopen if not!

[tommylees112]

Hi @bletham I'm having the same problem unfortunately. The MCMC sampler works for a few samples but then breaks after a while.

My code:

df = pd.read_csv('data/evi_mean.txt', delimiter=" ")

m = Prophet.Prophet(yearly_seasonality='auto', daily_seasonality=False, weekly_seasonality=False, mcmc_samples=300)
m.fit(df)

The data: evi_mean.txt

The error message:

/Users/TommyLees/anaconda3/envs/bayes/lib/python3.6/site-packages/pystan/misc.py:399: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  elif np.issubdtype(np.asarray(v).dtype, float):
/Users/TommyLees/anaconda3/envs/bayes/lib/python3.6/site-packages/pystan/misc.py:456: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
  if "chain_id" in inspect.getargspec(init).args:
----------------------------------------
RemoteTracebackTraceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/Users/TommyLees/anaconda3/envs/bayes/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/Users/TommyLees/anaconda3/envs/bayes/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "stanfit4anon_model_861b75c6337e237650a61ae58c4385ef_5050244804638337571.pyx", line 368, in stanfit4anon_model_861b75c6337e237650a61ae58c4385ef_5050244804638337571._call_sampler_star
  File "stanfit4anon_model_861b75c6337e237650a61ae58c4385ef_5050244804638337571.pyx", line 401, in stanfit4anon_model_861b75c6337e237650a61ae58c4385ef_5050244804638337571._call_sampler
ValueError: normal_lpdf: Scale parameter is 0, but must be > 0!
"""

[bletham]

@tommylees112 Thanks for the code to reproduce. I actually didn't run into this issue though - it does mention that issue early on in the sampling, but then is then able to sample well and completes without any issue.

Is there any more to your traceback? this is what I get when I run that code:

Gradient evaluation took 0.00031 seconds
1000 transitions using 10 leapfrog steps per transition would take 3.1 seconds.
Adjust your expectations accordingly!

Gradient evaluation took 0.000287 seconds
1000 transitions using 10 leapfrog steps per transition would take 2.87 seconds.
Adjust your expectations accordingly!

Gradient evaluation took 0.000187 seconds
1000 transitions using 10 leapfrog steps per transition would take 1.87 seconds.
Adjust your expectations accordingly!

Iteration:   1 / 300 [  0%]  (Warmup)
Iteration:   1 / 300 [  0%]  (Warmup)

Gradient evaluation took 0.00041 seconds
1000 transitions using 10 leapfrog steps per transition would take 4.1 seconds.
Adjust your expectations accordingly!

Iteration:   1 / 300 [  0%]  (Warmup)
Iteration:   1 / 300 [  0%]  (Warmup)
Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Exception: normal_lpdf: Scale parameter is 0, but must be > 0!  (in 'unknown file name' at line 112)

If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Exception: normal_lpdf: Scale parameter is 0, but must be > 0!  (in 'unknown file name' at line 112)

If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Exception: normal_lpdf: Scale parameter is 0, but must be > 0!  (in 'unknown file name' at line 112)

If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Exception: normal_lpdf: Scale parameter is 0, but must be > 0!  (in 'unknown file name' at line 112)

If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Iteration:  30 / 300 [ 10%]  (Warmup)
Iteration:  30 / 300 [ 10%]  (Warmup)
Iteration:  30 / 300 [ 10%]  (Warmup)
Iteration:  30 / 300 [ 10%]  (Warmup)
Iteration:  60 / 300 [ 20%]  (Warmup)
Iteration:  60 / 300 [ 20%]  (Warmup)
Iteration:  60 / 300 [ 20%]  (Warmup)
Iteration:  90 / 300 [ 30%]  (Warmup)
Iteration:  90 / 300 [ 30%]  (Warmup)
Iteration:  90 / 300 [ 30%]  (Warmup)
Iteration:  60 / 300 [ 20%]  (Warmup)
Iteration:  90 / 300 [ 30%]  (Warmup)
Iteration: 120 / 300 [ 40%]  (Warmup)
Iteration: 120 / 300 [ 40%]  (Warmup)
Iteration: 120 / 300 [ 40%]  (Warmup)
Iteration: 120 / 300 [ 40%]  (Warmup)
Iteration: 150 / 300 [ 50%]  (Warmup)
Iteration: 150 / 300 [ 50%]  (Warmup)
Iteration: 151 / 300 [ 50%]  (Sampling)
Iteration: 151 / 300 [ 50%]  (Sampling)
Iteration: 150 / 300 [ 50%]  (Warmup)
Iteration: 150 / 300 [ 50%]  (Warmup)
Iteration: 151 / 300 [ 50%]  (Sampling)
Iteration: 151 / 300 [ 50%]  (Sampling)
Iteration: 180 / 300 [ 60%]  (Sampling)
Iteration: 180 / 300 [ 60%]  (Sampling)
Iteration: 180 / 300 [ 60%]  (Sampling)
Iteration: 180 / 300 [ 60%]  (Sampling)
Iteration: 210 / 300 [ 70%]  (Sampling)
Iteration: 210 / 300 [ 70%]  (Sampling)
Iteration: 210 / 300 [ 70%]  (Sampling)
Iteration: 210 / 300 [ 70%]  (Sampling)
Iteration: 240 / 300 [ 80%]  (Sampling)
Iteration: 240 / 300 [ 80%]  (Sampling)
Iteration: 240 / 300 [ 80%]  (Sampling)
Iteration: 240 / 300 [ 80%]  (Sampling)
Iteration: 270 / 300 [ 90%]  (Sampling)
Iteration: 270 / 300 [ 90%]  (Sampling)
Iteration: 270 / 300 [ 90%]  (Sampling)
Iteration: 270 / 300 [ 90%]  (Sampling)
Iteration: 300 / 300 [100%]  (Sampling)

 Elapsed Time: 8.04412 seconds (Warm-up)
               7.66733 seconds (Sampling)
               15.7115 seconds (Total)

Iteration: 300 / 300 [100%]  (Sampling)

 Elapsed Time: 7.54096 seconds (Warm-up)
               7.52225 seconds (Sampling)
               15.0632 seconds (Total)

Iteration: 300 / 300 [100%]  (Sampling)

 Elapsed Time: 8.73468 seconds (Warm-up)
               8.01019 seconds (Sampling)
               16.7449 seconds (Total)

Iteration: 300 / 300 [100%]  (Sampling)

 Elapsed Time: 8.20928 seconds (Warm-up)
               7.52431 seconds (Sampling)
               15.7336 seconds (Total)

What version of pystan and fbprophet do you have installed?

[tommylees112]

Thanks for your reply Ben.

The full traceback below:

/Users/TommyLees/anaconda3/envs/bayes/lib/python3.6/site-packages/IPython/lib/pretty.py:91: DeprecationWarning: IPython.utils.signatures backport for Python 2 is deprecated in IPython 6, which only supports Python 3
  from IPython.utils.signatures import signature
/Users/TommyLees/anaconda3/envs/bayes/lib/python3.6/site-packages/pystan/misc.py:399: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  elif np.issubdtype(np.asarray(v).dtype, float):
/Users/TommyLees/anaconda3/envs/bayes/lib/python3.6/site-packages/pystan/misc.py:456: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
  if "chain_id" in inspect.getargspec(init).args:

Gradient evaluation took 0.000907 seconds
1000 transitions using 10 leapfrog steps per transition would take 9.07 seconds.
Adjust your expectations accordingly!

Gradient evaluation took 0.000818 seconds
1000 transitions using 10 leapfrog steps per transition would take 8.18 seconds.
Adjust your expectations accordingly!

Gradient evaluation took 0.000823 seconds
1000 transitions using 10 leapfrog steps per transition would take 8.23 seconds.
Adjust your expectations accordingly!

Gradient evaluation took 0.001165 seconds
1000 transitions using 10 leapfrog steps per transition would take 11.65 seconds.
Adjust your expectations accordingly!

Iteration:   1 / 300 [  0%]  (Warmup)
Iteration:   1 / 300 [  0%]  (Warmup)
Iteration:   1 / 300 [  0%]  (Warmup)
Iteration:   1 / 300 [  0%]  (Warmup)
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/Users/TommyLees/anaconda3/envs/bayes/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/Users/TommyLees/anaconda3/envs/bayes/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "stanfit4anon_model_861b75c6337e237650a61ae58c4385ef_5050244804638337571.pyx", line 368, in stanfit4anon_model_861b75c6337e237650a61ae58c4385ef_5050244804638337571._call_sampler_star
  File "stanfit4anon_model_861b75c6337e237650a61ae58c4385ef_5050244804638337571.pyx", line 401, in stanfit4anon_model_861b75c6337e237650a61ae58c4385ef_5050244804638337571._call_sampler
ValueError: normal_lpdf: Scale parameter is 0, but must be > 0!
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "mcmc_test.py", line 18, in <module>
    m.fit(df)
  File "/Users/TommyLees/anaconda3/envs/bayes/lib/python3.6/site-packages/fbprophet/forecaster.py", line 996, in fit
    **kwargs
  File "/Users/TommyLees/anaconda3/envs/bayes/lib/python3.6/site-packages/pystan/model.py", line 726, in sampling
    ret_and_samples = _map_parallel(call_sampler_star, call_sampler_args, n_jobs)
  File "/Users/TommyLees/anaconda3/envs/bayes/lib/python3.6/site-packages/pystan/model.py", line 81, in _map_parallel
    map_result = pool.map(function, args)
  File "/Users/TommyLees/anaconda3/envs/bayes/lib/python3.6/multiprocessing/pool.py", line 266, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/Users/TommyLees/anaconda3/envs/bayes/lib/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
ValueError: normal_lpdf: Scale parameter is 0, but must be > 0!

Version information:

(bayes) ➜  exploratory_drought_analysis python -c "import fbprophet; print(fbprophet.__version__)"        [system]
0.3

(bayes) ➜  exploratory_drought_analysis python -c "import pystan; print(pystan.__version__)"              [system]
2.17.1.0

Maybe I have my exceptions causing the programme to crash?

What's interesting is that I can compile and run other models written in stan.

[bletham]

Interesting, I'm not sure why it's raising the ValueError instead of just continuing sampling like it seems to in my system.

Can you try doing it with just 1 chain so it doesn't use multiprocessing?

m.fit(df, chains=1)

Besides that I'm not entirely sure what to do here. Maybe try upgrading pystan to 2.18, the latest version?

[tommylees112]

WOOOO thanks @bletham updating to Pystan 1.18 worked a treat.

Perfect! Thanks for your help!

[bletham]

Oh wonderful, that was a real shot in the dark! Odd that it was raising in 2.17.1, but glad it's working now.