search

shirley-wu / vdebugger

[EMNLP2024 Findings] VDebugger: Harnessing Execution Feedback for Debugging Visual Programs
https://shirley-wu.github.io/vdebugger/
Apache License 2.0
3 stars 1 forks source link

question about iteration #2

Closed aidialogue closed 3 weeks ago

aidialogue commented 1 month ago

Hello!You discussed the iteration numbers in paper, but I could not find any parameters related to the iteration counts in the provided code. Does the mention of iteration numbers imply that the infer_critic.py and infer_refine.py need to be executed multiple times?Would you kindly confirm if my understanding is correct? Could you please provide some guidance on how to properly set up and execute these iterations?Thanks！

shirley-wu commented 1 month ago

Hello, yes we need to run infer_critic.py and infer_refine.py for multiple times. To save time, we only run the code when it is changed after critic and refinement; that is, when the code is classified as incorrect (so it's necessary to be refined) and the refiner propose a change in the code. I'll update the relevant script next week, but it's pretty straightforward so you can also implement it by yourself

shirley-wu commented 3 weeks ago

Hi, sorry for the late response. I uploaded some helper scripts at https://github.com/shirley-wu/vdebugger/tree/main/vdebugger/iterative_helper. Hope that helps!