search

HumanCompatibleAI / overcooked_ai

A benchmark environment for fully cooperative human-AI performance.
https://arxiv.org/abs/1910.05789
MIT License
683 stars 144 forks source link

How to train human proxy model? #135

Closed call-me-anything-you-want closed 8 months ago

call-me-anything-you-want commented 8 months ago

This might be a silly question. But I can't find the source code for training a human proxy model. I noticed that overcooked_ai/src/human_aware_rl/imitation/reproduce_bc.py can train bc models which are then stored in imitation/bc_runs/train. But the overcooked_ai/src/human_aware_rl/ppo/evaluate.py requires models stored in imitation/bc_runs/test. Which file can I run to generate corresponding hp models?

jyan1999 commented 8 months ago

If I remember correctly, you just need to manually change the output directory and training dataset in overcooked_ai/src/human_aware_rl/imitation/reproduce_bc.py so it uses CLEAN_2019_HUMAN_DATA_TEST and saves the output model to the imitation/bc_runs/test dir. The human proxy model is basically another behavior cloning model trained with the held-out data to evaluate against other models at test time.

call-me-anything-you-want commented 8 months ago

@jyan1999 Thanks for the help, I've successfully obtained the human proxy models following your instructions.