Hello pymc team,
I bumped into some error in using pymc v4 or pymc v3 (see previous issue raised by me), so I tried running the rethinking notebook with pymc v5 to see if things would get better.
Computer environment:
Window11, anaconda, pymc v5
conda create -c conda-forge -n pymc_env "pymc>=5"
Libraries required for this notebook were installed as below:
pip install notebook, patsy, causalgraphicalmodels, daft, theanoconda install -c conda-forge watermark, seaborn, tqdm, aesara==2.8.7
Initially, I was hoping to install all the libraries according to the environment yaml file for pymc v4. However, there was some verion conflicts so I ended up installing some of the libraries via pip.
conda install -c conda-forge numpy=1.23
I have also downgraded numpy to avoid
Attributeerror: module has no attribute bool error
Overall I would summarised the errors as (1) unable convert tensor variable. from chapter 5,11,14,16 (2) ParallelSamplingError from chapter 7, 12 (3) others
Below are the errors I have found in those chapters:
chp5:
5.19
NotImplementedError Traceback (most recent call last)
Cell In[33], line 16
13 bM = pm.Normal("bM", 0, 0.5)
15 a = pm.Normal("a", 0, 0.2)
---> 16 mu = pm.Deterministic("mu", a + bA age_shared + bM marriage_shared)
NotImplementedError: Cannot convert bA to a tensor variable.
chp7
Code 7.16
RemoteTraceback Traceback (most recent call last)
RemoteTraceback:
"""
Traceback (most recent call last):
File "C:\Users\einat\anaconda3\envs\pymc_env\Lib\site-packages\pymc\sampling\parallel.py", line 112, in _unpickle_step_method
self._step_method = cloudpickle.loads(self._step_method)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: __generator_ctor() takes from 0 to 1 positional arguments but 2 were given
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\einat\anaconda3\envs\pymc_env\Lib\site-packages\pymc\sampling\parallel.py", line 120, in run
self._unpickle_step_method()
File "C:\Users\einat\anaconda3\envs\pymc_env\Lib\site-packages\pymc\sampling\parallel.py", line 114, in _unpickle_step_method
raise ValueError(unpickle_error)
ValueError: The model could not be unpickled. This is required for sampling with more than one core and multiprocessing context spawn or forkserver.
"""
The above exception was the direct cause of the following exception:
ValueError Traceback (most recent call last)
ValueError: The model could not be unpickled. This is required for sampling with more than one core and multiprocessing context spawn or forkserver.
The above exception was the direct cause of the following exception:
ParallelSamplingError: Chain 0 failed with: The model could not be unpickled. This is required for sampling with more than one core and multiprocessing context spawn or forkserver.
File ~\anaconda3\envs\pymc_env\Lib\site-packages\pymc\model.py:1972, in set_data(new_data, model, coords)
1969 model = modelcontext(model)
1971 for variable_name, new_value in new_data.items():
-> 1972 model.set_data(variable_name, new_value, coords=coords)
File ~\anaconda3\envs\pymc_env\Lib\site-packages\pymc\model.py:1135, in Model.set_data(self, name, values, coords)
1133 shared_object = self[name]
1134 if not isinstance(shared_object, SharedVariable):
-> 1135 raise TypeError(
1136 f"The variable {name} must be a SharedVariable"
1137 " (created through pm.MutableData() or pm.Data(mutable=True)) to allow updating. "
1138 f"The current type is: {type(shared_object)}"
1139 )
1141 if isinstance(values, list):
1142 values = np.array(values)
TypeError: The variable actor_id must be a SharedVariable (created through pm.MutableData() or pm.Data(mutable=True)) to allow updating. The current type is: <class 'pytensor.tensor.var.TensorConstant'>
Chp12:
Code 12.2
RemoteTraceback Traceback (most recent call last)
RemoteTraceback:
"""
Traceback (most recent call last):
File "C:\Users\einat\anaconda3\envs\pymc_env\Lib\site-packages\pymc\sampling\parallel.py", line 112, in _unpickle_step_method
self._step_method = cloudpickle.loads(self._step_method)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: __generator_ctor() takes from 0 to 1 positional arguments but 2 were given
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\einat\anaconda3\envs\pymc_env\Lib\site-packages\pymc\sampling\parallel.py", line 120, in run
self._unpickle_step_method()
File "C:\Users\einat\anaconda3\envs\pymc_env\Lib\site-packages\pymc\sampling\parallel.py", line 114, in _unpickle_step_method
raise ValueError(unpickle_error)
ValueError: The model could not be unpickled. This is required for sampling with more than one core and multiprocessing context spawn or forkserver.
"""
The above exception was the direct cause of the following exception:
ValueError Traceback (most recent call last)
ValueError: The model could not be unpickled. This is required for sampling with more than one core and multiprocessing context spawn or forkserver.
The above exception was the direct cause of the following exception:
ParallelSamplingError: Chain 0 failed with: The model could not be unpickled. This is required for sampling with more than one core and multiprocessing context spawn or forkserver.
File ~\anaconda3\envs\pymc_env\Lib\site-packages\arviz\stats\stats.py:1613, in waic(data, pointwise, var_name, scale, dask_kwargs)
1612 inference_data = convert_to_inference_data(data)
-> 1613 log_likelihood = _get_log_likelihood(inference_data, var_name=var_name)
1614 scale = rcParams["stats.ic_scale"] if scale is None else scale.lower()
File ~\anaconda3\envs\pymc_env\Lib\site-packages\arviz\stats\stats_utils.py:425, in get_log_likelihood(idata, var_name)
424 if not hasattr(idata, "log_likelihood"):
--> 425 raise TypeError("log likelihood not found in inference data object")
426 if var_name is None:
TypeError: log likelihood not found in inference data object
The above exception was the direct cause of the following exception:
File ~\anaconda3\envs\pymc_env\Lib\site-packages\arviz\stats\stats.py:455, in _calculate_ics(compare_dict, scale, ic, var_name)
454 except Exception as e:
--> 455 raise e.class(
456 f"Encountered error trying to compute {ic} from model {name}."
457 ) from e
458 return (compare_dict, scale, ic)
TypeError: Encountered error trying to compute waic from model m13_1.
The above exception was the direct cause of the following exception:
File ~\anaconda3\envs\pymc_env\Lib\site-packages\arviz\stats\stats.py:179, in compare(compare_dict, ic, method, b_samples, alpha, seed, scale, var_name)
177 (ics_dict, scale, ic) = _calculate_ics(compare_dict, scale=scale, ic=ic, var_name=var_name)
178 except Exception as e:
--> 179 raise e.class("Encountered error in ELPD computation of compare.") from e
180 names = list(ics_dict.keys())
181 if ic == "loo":
TypeError: Encountered error in ELPD computation of compare.
Chp14
Code 14.12
NotImplementedError Traceback (most recent call last)
Cell In[15], line 12
9 a = pm.Normal("a", 5.0, 2.0) # prior for average intercept
10 b = pm.Normal("b", -1.0, 0.5) # prior for average slope
11 ab_cafe = pm.MvNormal(
---> 12 "ab_cafe", mu=at.stack([a, b]), chol=chol, shape=(N_cafes, 2)
13 ) # population of varying effects
14 # shape needs to be (N_cafes, 2) because we're getting back both a and b for each cafe
16 mu = ab_cafe[cafe_idx, 0] + ab_cafe[cafe_idx, 1] * d["afternoon"].values # linear model
File ~\anaconda3\envs\pymc_env\Lib\site-packages\aesara\tensor\basic.py:2619, in stack(*tensors, *kwargs)
2617 dtype = aes.upcast([i.dtype for i in tensors])
2618 return MakeVector(dtype)(tensors)
-> 2619 return join(axis, [shape_padaxis(t, axis) for t in tensors])
File ~\anaconda3\envs\pymc_env\Lib\site-packages\aesara\tensor\basic.py:2619, in (.0)
2617 dtype = aes.upcast([i.dtype for i in tensors])
2618 return MakeVector(dtype)(tensors)
-> 2619 return join(axis, *[shape_padaxis(t, axis) for t in tensors])
File ~\anaconda3\envs\pymc_env\Lib\site-packages\aesara\tensor\shape.py:846, in shape_padaxis(t, axis)
824 def shape_padaxis(t, axis):
825 """Reshape t by inserting 1 at the dimension axis.
826
827 Examples
(...)
844
845 """
--> 846 _t = at.as_tensor_variable(t)
848 ndim = _t.ndim + 1
849 if not -ndim <= axis < ndim:
File ~\anaconda3\envs\pymc_env\Lib\site-packages\aesara\tensor__init__.py:49, in as_tensor_variable(x, name, ndim, kwargs)
17 def as_tensor_variable(
18 x: TensorLike, name: Optional[str] = None, ndim: Optional[int] = None, kwargs
19 ) -> "TensorVariable":
20 """Convert x into an equivalent TensorVariable.
21
22 This function can be used to turn ndarrays, numbers, ScalarType instances,
(...)
47
48 """
---> 49 return _as_tensor_variable(x, name, ndim, **kwargs)
File ~\anaconda3\envs\pymc_env\Lib\functools.py:909, in singledispatch..wrapper(*args, *kw)
905 if not args:
906 raise TypeError(f'{funcname} requires at least '
907 '1 positional argument')
--> 909 return dispatch(args[0].class)(args, **kw)
NotImplementedError: Cannot convert a to a tensor variable.---------------------------------------------------------------------------
NotImplementedError Traceback (most recent call last)
Cell In[15], line 12
9 a = pm.Normal("a", 5.0, 2.0) # prior for average intercept
10 b = pm.Normal("b", -1.0, 0.5) # prior for average slope
11 ab_cafe = pm.MvNormal(
---> 12 "ab_cafe", mu=at.stack([a, b]), chol=chol, shape=(N_cafes, 2)
13 ) # population of varying effects
14 # shape needs to be (N_cafes, 2) because we're getting back both a and b for each cafe
16 mu = ab_cafe[cafe_idx, 0] + ab_cafe[cafe_idx, 1] * d["afternoon"].values # linear model
File ~\anaconda3\envs\pymc_env\Lib\site-packages\aesara\tensor\basic.py:2619, in stack(*tensors, *kwargs)
2617 dtype = aes.upcast([i.dtype for i in tensors])
2618 return MakeVector(dtype)(tensors)
-> 2619 return join(axis, [shape_padaxis(t, axis) for t in tensors])
File ~\anaconda3\envs\pymc_env\Lib\site-packages\aesara\tensor\basic.py:2619, in (.0)
2617 dtype = aes.upcast([i.dtype for i in tensors])
2618 return MakeVector(dtype)(tensors)
-> 2619 return join(axis, *[shape_padaxis(t, axis) for t in tensors])
File ~\anaconda3\envs\pymc_env\Lib\site-packages\aesara\tensor\shape.py:846, in shape_padaxis(t, axis)
824 def shape_padaxis(t, axis):
825 """Reshape t by inserting 1 at the dimension axis.
826
827 Examples
(...)
844
845 """
--> 846 _t = at.as_tensor_variable(t)
848 ndim = _t.ndim + 1
849 if not -ndim <= axis < ndim:
File ~\anaconda3\envs\pymc_env\Lib\site-packages\aesara\tensor__init__.py:49, in as_tensor_variable(x, name, ndim, kwargs)
17 def as_tensor_variable(
18 x: TensorLike, name: Optional[str] = None, ndim: Optional[int] = None, kwargs
19 ) -> "TensorVariable":
20 """Convert x into an equivalent TensorVariable.
21
22 This function can be used to turn ndarrays, numbers, ScalarType instances,
(...)
47
48 """
---> 49 return _as_tensor_variable(x, name, ndim, **kwargs)
File ~\anaconda3\envs\pymc_env\Lib\functools.py:909, in singledispatch..wrapper(*args, *kw)
905 if not args:
906 raise TypeError(f'{funcname} requires at least '
907 '1 positional argument')
--> 909 return dispatch(args[0].class)(args, **kw)
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
File ~\anaconda3\envs\pymcenv\Lib\site-packages\theano\tensor\basic.py:246, in constant(x, name, ndim, dtype)
245 try:
--> 246 ttype = TensorType(dtype=x.dtype, broadcastable=bcastable)
247 if not constant.enable:
File ~\anaconda3\envs\pymc_env\Lib\site-packages\theano\tensor\type.py:51, in TensorType.init(self, dtype, broadcastable, name, sparse_grad)
50 self.broadcastable = tuple(bool(b) for b in broadcastable)
---> 51 self.dtype_specs() # error checking is done there
52 self.name = name
File ~\anaconda3\envs\pymc_env\Lib\site-packages\theano\tensor\type.py:271, in TensorType.dtype_specs(self)
270 except KeyError:
--> 271 raise TypeError("Unsupported dtype for %s: %s"
272 % (self.class.name, self.dtype))
TypeError: Unsupported dtype for TensorType: object
During handling of the above exception, another exception occurred:
File ~\anaconda3\envs\pymc_env\Lib\site-packages\theano\tensor\basic.py:266, in constant(x, name, ndim, dtype)
265 except Exception:
--> 266 raise TypeError("Could not convert %s to TensorType" % x, type(x))
TypeError: ('Could not convert p to TensorType', <class 'pytensor.tensor.var.TensorVariable'>)
During handling of the above exception, another exception occurred:
AsTensorError Traceback (most recent call last)
Cell In[5], line 5
3 k = pm.Exponential("k", 0.5)
4 p = pm.Beta("p", alpha=2, beta=18)
----> 5 mu = pm.Deterministic("mu", pm.math.log(np.pi k tt.pow(p, 2) * tt.pow(d.h, 3)))
6 w = pm.Lognormal("w", mu, sigma, observed=d.w)
File ~\anaconda3\envs\pymc_env\Lib\site-packages\theano\gof\op.py:615, in PureOp.call(self, *inputs, *kwargs)
584 """
585 Optional: return some or all output[s] of make_node.
586
(...)
612
613 """
614 return_list = kwargs.pop('return_list', False)
--> 615 node = self.make_node(inputs, **kwargs)
617 if config.compute_test_value != 'off':
618 run_perform = True
File ~\anaconda3\envs\pymc_env\Lib\site-packages\theano\tensor\elemwise.py:480, in Elemwise.make_node(self, inputs)
474 def make_node(self, inputs):
475 """
476 If the inputs have different number of dimensions, their shape
477 is left-completed to the greatest number of dimensions with 1s
478 using DimShuffle.
479 """
--> 480 inputs = list(map(as_tensor_variable, inputs))
481 out_dtypes, out_broadcastables, inputs = self.get_output_info(
482 DimShuffle, *inputs)
483 outputs = [TensorType(dtype=dtype, broadcastable=broadcastable)()
484 for dtype, broadcastable in izip(out_dtypes,
485 out_broadcastables)]
Hello pymc team, I bumped into some error in using pymc v4 or pymc v3 (see previous issue raised by me), so I tried running the rethinking notebook with pymc v5 to see if things would get better. Computer environment: Window11, anaconda, pymc v5
conda create -c conda-forge -n pymc_env "pymc>=5"
Libraries required for this notebook were installed as below:pip install notebook, patsy, causalgraphicalmodels, daft, theano
conda install -c conda-forge watermark, seaborn, tqdm, aesara==2.8.7
Initially, I was hoping to install all the libraries according to the environment yaml file for pymc v4. However, there was some verion conflicts so I ended up installing some of the libraries via pip.conda install -c conda-forge numpy=1.23
I have also downgraded numpy to avoidOverall I would summarised the errors as (1) unable convert tensor variable. from chapter 5,11,14,16 (2) ParallelSamplingError from chapter 7, 12 (3) others
Below are the errors I have found in those chapters: chp5: 5.19
chp7 Code 7.16
chp8 Code 8.1:
Chp11 Code 11.16 and 11.17
Chp12: Code 12.2
Chp13 code 13.4
Chp14 Code 14.12
Chp16 code 16.2
Please feel free to let me know if you have any thoughts