search

noahshinn / reflexion

[NeurIPS 2023] Reflexion: Language Agents with Verbal Reinforcement Learning
MIT License
2.46k stars 240 forks source link

Which commit matches your published work (for HumanEval-rs)? #23

Open geekoftheweek opened 1 year ago

geekoftheweek commented 1 year ago

I'd like to run your code as it was implemented for the original paper. Would it be possible to add a link in the README that points to the specific commit representing the code used to achieve your results as published in the Reflexion paper? Thanks!

geekoftheweek commented 1 year ago

Critical reading failure on my part -- I see now that you do reference how to find the original code. Closing this. Thank you!

geekoftheweek commented 1 year ago

Reopening, as I can't find any code in reflexion-draft that references your published results against HumanEval-rs.

noahshinn commented 1 year ago

Hi @geekoftheweek , you can use the run script here https://github.com/noahshinn024/reflexion/blob/main/programming_runs/run_reflexion.sh. The only changes have been refactorings on the original code. Let me know if there are any questions

geekoftheweek commented 1 year ago

Hmm. Perhaps you can help me understand, then, why these two files have different prompt text?

ai-nikolai commented 10 months ago

@noahshinn @geekoftheweek thanks for this thread.

Are there any updates on the above.

Also, I am struggling to understand the results on alfworld (and how to reproduce them), as e.g. env 1-3 all seem to have been successful for you, however, it never reaches success if running the script. (Unless the order of envs is different?)

Thank you.