vyokky
commented
9 months ago Hi! Thx so much for this awesome work :) I really liked the paper!
In general I am big fan of the decompositional approaches for reasoning using tree-based algorithms. One question I was curious about is why you consider ToT less flexible? From your paper I did not get that you are aggregating states, or is that the case?
Once again, appreciate our work!
Another short question would be regarding the revision: you mentioned that you have one LLM invocation that contains all the trajectories for the current solution you look at. Could you release an example prompt for either of your benchmarks; I am specifically interested how scalable those are. I have discovered the training output files in your repo, maybe I am just not finding such examples.
Hi, many thanks for your interest to our work! The flexiblility in our work refer to whether the framework can generate thought structure with any topolopy, including graphs. ToT IMO, are limited to trees or chains, if I understand correctly.
For the revision, I attach a prompt below. Help this can help:
Input:
Several prompts above ...
For current problem
If the final answer is not exactly 24, then the corresponding [Steps] is considered [wrong]. Please help me identify the exact wrong step based on its left number, among [Step 1, Step 2, Step 3]. Input: 2 9 10 12 Steps: [Steps 1] 12 2 = 24 (left: 9 10 24) Expression: 9, 10, (12) (2) [Steps 2] 24 - 10 = 14 (left: 9 14) Expression: 9, ((12) (2)) - (10) [Steps 3] 9 + 14 = 23 (left: 23) Expression: (9) + ((12) (2)) - (10) The Steps are wrong. Because it can not reach 24 in the end. To be specific,
Output:
23 is not equal to 24. [Steps 2] is wrong. Because it is impossible to reach 24 from the step 2. After Step 2, left numbers are 9, 14. 9 + 14 = 23 9 * 14 = 126 9 - 14 = -5 It is impossible to reach 24 from [Steps 2].
Hi! Thx so much for this awesome work :) I really liked the paper!
In general I am big fan of the decompositional approaches for reasoning using tree-based algorithms. One question I was curious about is why you consider ToT less flexible? From your paper I did not get that you are aggregating states, or is that the case?
Once again, appreciate our work!
Another short question would be regarding the revision: you mentioned that you have one LLM invocation that contains all the trajectories for the current solution you look at. Could you release an example prompt for either of your benchmarks; I am specifically interested how scalable those are. I have discovered the training output files in your repo, maybe I am just not finding such examples.