Closed fbarfi closed 1 year ago
I am not able to reproduce this error locally. Maybe this is related to https://github.com/fastai/fastprogress/issues/32
Thanks aloctavodia for the suggestion. I followed the link and checked the suggestion in it about reinstalling ipywidgets which I did. It does not solve the problem. I am running the latest ipywidgets 7.7.0. I tried to run the notebook on collar but then was faced with other issues (after I installed pymc there).
Thanks for checking. I will check this tomorrow, in the meantime could you try installing the latest version (pymc 4.0). https://github.com/pymc-devs/pymc/wiki/Installation-Guide-(MacOS)
Thanks. I have already installed pymc : 4.0.0b6. Isn't that the latest?
almost, we have recently released the first stable version of the 4.x series. If you install from main or following the intructions in the link i posterd you will get the version 4.0.
will do. thanks.
Just reinstalled everything afresh as directed. Yet getting the same error. thanks.
Error about HTML disappears when I do:
pm.sample_smc(progressbar=False)
Now I get the following error:
---------------------------------------------------------------------------
RemoteTraceback Traceback (most recent call last)
RemoteTraceback:
"""
Traceback (most recent call last):
File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/multiprocessing/pool.py", line 51, in starmapstar
return list(itertools.starmap(args[0], args[1]))
File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/smc/sample_smc.py", line 439, in _apply_args_and_kwargs
return fn(*args, **kwargs)
File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/smc/sample_smc.py", line 363, in _sample_smc_int
smc._initialize_kernel()
File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/smc/smc.py", line 225, in _initialize_kernel
initial_point, [self.model.datalogpt], self.variables, shared
File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/model.py", line 847, in datalogpt
return self.observedlogpt + self.potentiallogpt
File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/model.py", line 864, in observedlogpt
return self.logpt(vars=self.observed_RVs)
File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/model.py", line 745, in logpt
rv_logps = joint_logpt(list(rv_values.keys()), rv_values, sum=False, jacobian=jacobian)
File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/distributions/logprob.py", line 226, in joint_logpt
temp_logp_var_dict = factorized_joint_logprob(
File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/aeppl/joint_logprob.py", line 147, in factorized_joint_logprob
q_logprob_vars = _logprob(
File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/functools.py", line 889, in wrapper
return dispatch(args[0].__class__)(*args, **kw)
File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/aeppl/logprob.py", line 85, in _logprob
raise NotImplementedError(f"Logprob method not implemented for {op}")
NotImplementedError: Logprob method not implemented for Simulator_rv{0, (0, 0), floatX, False}
"""
The above exception was the direct cause of the following exception:
NotImplementedError Traceback (most recent call last)
Input In [6], in <cell line: 1>()
3 b = pm.HalfNormal("b", sigma=1)
4 s = pm.Simulator("s", normal_sim, params=(a, b),
5 sum_stat="sort", epsilon=1, observed=data)
----> 7 idata = pm.sample_smc(progressbar=False)
8 idata.extend(pm.sample_posterior_predictive(idata))
File /opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/smc/sample_smc.py:227, in sample_smc(draws, kernel, start, model, random_seed, chains, cores, compute_convergence_checks, return_inferencedata, idata_kwargs, progressbar, **kernel_kwargs)
224 t1 = time.time()
226 if cores > 1:
--> 227 results = run_chains_parallel(
228 chains, progressbar, _sample_smc_int, params, random_seed, kernel_kwargs, cores
229 )
230 else:
231 results = run_chains_sequential(
232 chains, progressbar, _sample_smc_int, params, random_seed, kernel_kwargs
233 )
File /opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/smc/sample_smc.py:408, in run_chains_parallel(chains, progressbar, to_run, params, random_seed, kernel_kwargs, cores)
406 params = tuple(cloudpickle.dumps(p) for p in params)
407 kernel_kwargs = {key: cloudpickle.dumps(value) for key, value in kernel_kwargs.items()}
--> 408 results = _starmap_with_kwargs(
409 pool,
410 to_run,
411 [(*params, random_seed[chain], chain, pbars[chain]) for chain in range(chains)],
412 repeat(kernel_kwargs),
413 )
414 results = tuple(cloudpickle.loads(r) for r in results)
415 pool.close()
File /opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/site-packages/pymc/smc/sample_smc.py:435, in _starmap_with_kwargs(pool, fn, args_iter, kwargs_iter)
431 def _starmap_with_kwargs(pool, fn, args_iter, kwargs_iter):
432 # Helper function to allow kwargs with Pool.starmap
433 # Copied from https://stackoverflow.com/a/53173433/13311693
434 args_for_starmap = zip(repeat(fn), args_iter, kwargs_iter)
--> 435 return pool.starmap(_apply_args_and_kwargs, args_for_starmap)
File /opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/multiprocessing/pool.py:372, in Pool.starmap(self, func, iterable, chunksize)
366 def starmap(self, func, iterable, chunksize=None):
367 '''
368 Like `map()` method but the elements of the `iterable` are expected to
369 be iterables as well and will be unpacked as arguments. Hence
370 `func` and (a, b) becomes func(a, b).
371 '''
--> 372 return self._map_async(func, iterable, starmapstar, chunksize).get()
File /opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.10/multiprocessing/pool.py:771, in ApplyResult.get(self, timeout)
769 return self._value
770 else:
--> 771 raise self._value
NotImplementedError: Logprob method not implemented for Simulator_rv{0, (0, 0), floatX, False}
Can you try running with a single core? pm.sample_smc(cores=1)
or something.
Also, please provide the whole snippet you are using so that we can try to reproduce it locally.
this is the example I was running from the examples posted on pymc
[SMC-ABC_Lotka-Volterra_example.ipynb](https://github.com/pymc-devs/pymc-examples/blob/main/examples/samplers/SMC-ABC_Lotka-Volterra_example.ipynb)
I did what you suggested with cores=1 and everything works perfectly. Does this mean that one cannot use more than one core? Thanks.
It should work, but there seems to be an issue with multiprocessing on your end :/
I am not sure what to say. The moment used cores =2 I get the message at the end as
NotImplementedError: Logprob method not implemented for Simulator_rv{0, (0, 0), floatX, False}
it does not mention anywhere the multiprocessing. Could it be a macOS issue, or perhaps an apple M1 issue? thanks.
I am looking into the other examples for samplers that use the pm.sample_smc and they all work with cores=1 and multiple chains (4). thanks.
The traceback above indicates Python 3.10. Are we sure that Aesara/PyMC work with 3.10?
Because #5209
Yes they do work. I ve tried many codes, including even with sampling_jax.
I have installed PyMC v5 on a new environment and received the same error when running the Bayes Factor notebook.
I updated the code to: ...progressbar=False)
and the error was solved.
I have tried running the notebook to reproduce the error but in my case it worked, so it isn't yet clear how to reproduce this issue :/
I also get this error running the SMC-ABC example from the docs:
import numpy as np
import pymc as pm
import matplotlib.pyplot as plt
import arviz as az
data = np.random.normal(loc=0, scale=1, size=1000)
def normal_sim(rng, a, b, size=1000):
return rng.normal(a, b, size=size)
with pm.Model() as example:
a = pm.Normal("a", mu=0, sigma=5)
b = pm.HalfNormal("b", sigma=1)
s = pm.Simulator("s", normal_sim, params=(a, b), sum_stat="sort", epsilon=1, observed=data)
idata = pm.sample_smc()
idata.extend(pm.sample_posterior_predictive(idata))
The error goes away (no other errors) when I add the progressbar=False
argument to sample_smc
.
I'm using version 5.0.2 in an Anaconda environment on Mac OS X with Python 3.11.
Here is the package list:
Description of your problem
Running the example: SMC-ABC_Lotka-Volterra_example. exactly as is. Getting the error down below. Thanks!
Please provide the full traceback.
Complete error traceback
```python RemoteTraceback Traceback (most recent call last) RemoteTraceback: """ Traceback (most recent call last): File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/multiprocessing/pool.py", line 51, in starmapstar return list(itertools.starmap(args[0], args[1])) File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/site-packages/pymc/smc/sample_smc.py", line 439, in _apply_args_and_kwargs return fn(*args, **kwargs) File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/site-packages/pymc/smc/sample_smc.py", line 361, in _sample_smc_int progressbar.update_bar(getattr(progressbar, "offset", 0) + 0) File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/site-packages/fastprogress/fastprogress.py", line 76, in update_bar else: self.on_update(val, f'{100 * val/self.total:.2f}% [{val}/{self.total} {elapsed_t}<{remaining_t}{end}]') File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_4/lib/python3.10/site-packages/fastprogress/fastprogress.py", line 127, in on_update if self.display: self.out.update(HTML(self.progress)) NameError: name 'HTML' is not defined """ The above exception was the direct cause of the following exception: NameError Traceback (most recent call last) Input In [5], inPlease provide any additional information below.
Versions and main components
Python implementation: CPython Python version : 3.10.4 IPython version : 8.4.0
Compiler : Clang 12.0.1 OS : Darwin Release : 21.5.0 Machine : arm64 Processor : arm CPU cores : 10 Architecture: 64bit
arviz : 0.12.1 matplotlib: 3.5.2 numpy : 1.22.4 pymc : 4.0.0b6