search

hddm-devs / hddm

HDDM is a python module that implements Hierarchical Bayesian parameter estimation of Drift Diffusion Models (via PyMC).
http://ski.clps.brown.edu/hddm_docs/
Other
268 stars 119 forks source link

[Compatibility issues with Kabuki] Issue with Fixed sampling_method="cssm" Causing PPC Errors in HDDM Regression Models #119

Open panwanke opened 1 week ago

panwanke commented 1 week ago

Problem Description:

We identified an issue in the hddm_regression module where the fixed sampling_method="cssm" parameter causes PPC (Posterior Predictive Check) errors in dockerHDDM InferenceData (.nc file) not generated for HDDMRegression with stimulus coding.

Specifically, the problem arises in the following line of code: https://github.com/hddm-devs/hddm/blob/6e766ef315629c20cd0be7267555c90c39cc0446/hddm/models/hddm_regression.py#L130

Here, the wfpt_reg_like function is defined with sampling_method="cssm". This setting is hardcoded and cannot be modified during model definition, leading to PPC errors in specific scenarios. In contrast, the default sampling_method="drift" does not result in such issues in the version of hddm 0.8.0, see InferenceData (.nc file) not generated for HDDMRegression with stimulus coding.

Steps to Reproduce:

  1. Fit a regression model using stimcoding as a regressor with the current HDDM version (>=1.0.0).
  2. Perform PPC analysis, which reveals errors due to the fixed sampling_method="cssm".

Suggested Fix:

We recommend allowing users to configure the sampling_method parameter during model definition to override the hardcoded sampling_method="cssm". Alternatively, reverting to the default sampling_method="drift" may resolve the PPC errors.