search

ThomasAkam / Two-step_ACC

Analysis code for Akam et al. Neuron 2021 - The anterior cingulate cortex predicts future states to mediate model-based action selection.
GNU General Public License v3.0
3 stars 1 forks source link

Question about agent gradient #1

Closed cyf203 closed 1 month ago

cyf203 commented 1 month ago

I am trying to adapt your model_fitting.py for my choice data set. Your code is well-written. I am puzzling about "jac=agent.calculates_gradient”, at line 149-150: fit = op.minimize(fit_func, init_params_U, jac=agent.calculates_gradient, options={'disp': verbose, 'gtol': 1e-7}) because when I check your two-step agent code, there is no such property called "calculates_gradient". Appreciate for a bit clarification here.

Also, just a confirmation, did you use gradient function in model fitting?

Thank you!

ThomasAkam commented 1 month ago

Hi @cyf203,

The calculates_gradients property is set to False in the RL_agent class that all the different individual RL agents subclass. None of the RL agents calculate an analytical likelihood gradient for the model fitting, though the logistic regression model does (see its session_likelihood function). I used the same hierarchical model fitting code for both the RL and logistic regression modelling in this paper, though for mixed effects logistic regression there are standard packages (e.g. the glmer function in the lme4 R package) which are probably a better solution.

cyf203 commented 1 month ago

Thank you!