search

princeton-nlp / intercode

[NeurIPS 2023 D&B] Code repository for InterCode benchmark https://arxiv.org/abs/2306.14898
https://intercode-benchmark.github.io/
MIT License
194 stars 32 forks source link

HumanPolicy #1

Closed eyuansu62 closed 1 year ago

eyuansu62 commented 1 year ago

In the run_sql.py, you initialize the pocily by policy = HumanPolicy(). Shouldn't it be ChatGPTPolicy?

john-b-yang commented 1 year ago

Hello! Thanks for your comment.

The run_<code>.py files are more meant to be demonstrations that allow you to directly interact with InterCode environments.

It is not intended to showcase how to run language models end to end on these environments. If you're looking for examples, please check out the experiments folder and the wiki for a more high level overview.

Hope this helps! If you have any additional comments, feel free to follow up. Otherwise, I'll close this in a day.