AllenNeuralDynamics / aind-dynamic-foraging-models

behavioral models for the dynamic foraging task
MIT License
0 stars 0 forks source link

feat: migrate Han's RL-MLE fittings (still working in progress!) #18

Closed hanhou closed 2 months ago

hanhou commented 3 months ago

Steps to refactor:

hanhou commented 2 months ago

I decided to refactor the code so that agent, environment, and task are fully disentangled.

image

Here are some ideas,

  1. We code all AIND behavioral tasks (including future VR foraging / force foraging) using gymnasium in a shared library aind-behavior-gym. For dynamic foraging, we can further separate the block structure logic to a separate library called aind-dynamic-foraging-reward-schedule

  2. Both aind-behavior-gym and the behavioral GUI that trains animal call the same task class in reward-schedule , so that our artificial agents and animals are doing exactly the same task. 

  3. On the other hand, we can have different agent libraries that perform the task in the gym. Could be either Po-Chen's meta-rl agents, my Q-learning models, Lukasz's Bayesian agents, or Ulises's RNNs.

  4. If we were to fit the agents to animal behavior, we can write the fitting functions inside the agent libraries since fitting should not rely on the exact task.

@rachelstephlee @ZhixiaoSu @alexpiet

hanhou commented 2 months ago

To prevent this PR from growing too big, I'll go ahead and merge it to develop, and then work on remaining steps.