kyegomez / tree-of-thoughts

Plug in and Play Implementation of Tree of Thoughts: Deliberate Problem Solving with Large Language Models that Elevates Model Reasoning by atleast 70%
https://discord.gg/qUtxnK2NMf
Apache License 2.0
4.29k stars 361 forks source link

bad experience #25

Closed novohool closed 1 year ago

novohool commented 1 year ago

not work

use 20,20,2,15 and basic arithmetic operations (+-*/) to obtain 24
kyegomez commented 1 year ago

What's the output of the terminal?

yhyu13 commented 1 year ago

@jingslunt You would actually need gpt-4 to better solve this problem, and it is quite slow

novohool commented 1 year ago

@jingslunt You would actually need gpt-4 to better solve this problem, and it is quite slow

it works for you? I am chatgptplus user,but can't access to use gpt4 api🥹

kyegomez commented 1 year ago

GPT35 works just fine

novohool commented 1 year ago

GPT35 works just fine

% python3 example.py
Using api_model gpt-3.5-turbo
We receive a state of type <class 'str'> For state:  use 20,20,2,15  and basic arithmetic operations (+-*/) to obtain 24 

Considering the thoughts you've had until now:

use 20,20,2,15  and basic arithmetic operations (+-*/) to obtain 24

Devise the next coherent thought that will aid in advancing the reasoning process and achieving a solution to use 20,20,2,15  and basic arithmetic operations (+-*/) to obtain 24. Assess various scenarios, think unconventionally, anticipate potential challenges, and resolve any outstanding queries. Tap into your mind's full potential and make certain no open questions remain.Write down your observations in format 'Observation:xxxx', then write down your thoughts in format 'Thoughts:xxxx'.
thoughts: ['Observation: The numbers given have a wide range of values and only one of them is odd. \n\nThoughts: Maybe I can use the fact that 15 is an odd number to my advantage. I could try subtracting it from one of']
thoughts: ['Observation: The numbers given have a wide range of values and only one of them is odd. \n\nThoughts: Maybe I can use the fact that 15 is an odd number to my advantage. I could try subtracting it from one of', 'Observation: \n- We have two numbers that are the same (20) and two different numbers (2 and 15).\n- The target number is 24, which is smaller than the sum of the two 20s.\n\nThoughts:\n']
thoughts: ['Observation: The numbers given have a wide range of values and only one of them is odd. \n\nThoughts: Maybe I can use the fact that 15 is an odd number to my advantage. I could try subtracting it from one of', 'Observation: \n- We have two numbers that are the same (20) and two different numbers (2 and 15).\n- The target number is 24, which is smaller than the sum of the two 20s.\n\nThoughts:\n', 'Observation: The numbers given are all even except for 15.\nThoughts: Maybe we can use this fact to our advantage and try to make an odd number out of the even numbers and then combine it with 15 to get 24.']
thoughts: ['Observation: The numbers given have a wide range of values and only one of them is odd. \n\nThoughts: Maybe I can use the fact that 15 is an odd number to my advantage. I could try subtracting it from one of', 'Observation: \n- We have two numbers that are the same (20) and two different numbers (2 and 15).\n- The target number is 24, which is smaller than the sum of the two 20s.\n\nThoughts:\n', 'Observation: The numbers given are all even except for 15.\nThoughts: Maybe we can use this fact to our advantage and try to make an odd number out of the even numbers and then combine it with 15 to get 24.', 'Observation: The numbers given have a wide range of values and none of them are close to 24.\nThoughts: It might be helpful to break down the problem into smaller parts and find ways to manipulate the numbers to get closer to 24']
thoughts: ['Observation: The numbers given have a wide range of values and only one of them is odd. \n\nThoughts: Maybe I can use the fact that 15 is an odd number to my advantage. I could try subtracting it from one of', 'Observation: \n- We have two numbers that are the same (20) and two different numbers (2 and 15).\n- The target number is 24, which is smaller than the sum of the two 20s.\n\nThoughts:\n', 'Observation: The numbers given are all even except for 15.\nThoughts: Maybe we can use this fact to our advantage and try to make an odd number out of the even numbers and then combine it with 15 to get 24.', 'Observation: The numbers given have a wide range of values and none of them are close to 24.\nThoughts: It might be helpful to break down the problem into smaller parts and find ways to manipulate the numbers to get closer to 24', 'Observation: The numbers given have a wide range of values and only one of them is odd.\n\nThoughts: Perhaps we can use the fact that only one of the numbers is odd to our advantage. We could try adding or subtracting the odd']
Generated thoughts: ['Observation: The numbers given have a wide range of values and only one of them is odd. \n\nThoughts: Maybe I can use the fact that 15 is an odd number to my advantage. I could try subtracting it from one of', 'Observation: \n- We have two numbers that are the same (20) and two different numbers (2 and 15).\n- The target number is 24, which is smaller than the sum of the two 20s.\n\nThoughts:\n', 'Observation: The numbers given are all even except for 15.\nThoughts: Maybe we can use this fact to our advantage and try to make an odd number out of the even numbers and then combine it with 15 to get 24.', 'Observation: The numbers given have a wide range of values and none of them are close to 24.\nThoughts: It might be helpful to break down the problem into smaller parts and find ways to manipulate the numbers to get closer to 24', 'Observation: The numbers given have a wide range of values and only one of them is odd.\n\nThoughts: Perhaps we can use the fact that only one of the numbers is odd to our advantage. We could try adding or subtracting the odd']
We receive a state of type <class 'str'> For state:  Observation: 
- We have two numbers that are the same (20) and two different numbers (2 and 15).
- The target number is 24, which is smaller than the sum of the two 20s.

Thoughts:

2023-05-28 13:43:54,724 - INFO - error_code=None error_message='That model is currently overloaded with other requests. You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the request ID b38f8e0c68ba204e3d99ee84e58bb4a9 in your message.)' error_param=None error_type=server_error message='OpenAI API error received' stream_error=False
That model is currently overloaded with other requests. You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the request ID b38f8e0c68ba204e3d99ee84e58bb4a9 in your message.), sleep for 30s, set it by env OPENAI_RATE_TIMEOUT
state: 0.25
value: 0.25
2023-05-28 13:44:26,366 - INFO - State: Observation: 
- We have two numbers that are the same (20) and two different numbers (2 and 15).
- The target number is 24, which is smaller than the sum of the two 20s.

Thoughts:
, Value: 0.25
We receive a state of type <class 'str'> For state:  Observation: The numbers given are all even except for 15.
Thoughts: Maybe we can use this fact to our advantage and try to make an odd number out of the even numbers and then combine it with 15 to get 24. 

state: 0.65
value: 0.65
2023-05-28 13:44:27,595 - INFO - State: Observation: The numbers given are all even except for 15.
Thoughts: Maybe we can use this fact to our advantage and try to make an odd number out of the even numbers and then combine it with 15 to get 24., Value: 0.65
We receive a state of type <class 'tuple'> For state:  ('use 20,20,2,15  and basic arithmetic operations (+-*/) to obtain 24', 'Observation: The numbers given are all even except for 15.\nThoughts: Maybe we can use this fact to our advantage and try to make an odd number out of the even numbers and then combine it with 15 to get 24.') 

Considering the thoughts you've had until now:

use 20,20,2,15  and basic arithmetic operations (+-*/) to obtain 24
Observation: The numbers given are all even except for 15.
Thoughts: Maybe we can use this fact to our advantage and try to make an odd number out of the even numbers and then combine it with 15 to get 24.

Devise the next coherent thought that will aid in advancing the reasoning process and achieving a solution to use 20,20,2,15  and basic arithmetic operations (+-*/) to obtain 24. Assess various scenarios, think unconventionally, anticipate potential challenges, and resolve any outstanding queries. Tap into your mind's full potential and make certain no open questions remain.Write down your observations in format 'Observation:xxxx', then write down your thoughts in format 'Thoughts:xxxx'.
thoughts: ['Observation: The sum of the four given numbers is 57.\nThoughts: Since we need to use all four numbers to obtain 24, we can try to manipulate the numbers in a way that the sum of the resulting equation is close to']
thoughts: ['Observation: The sum of the four given numbers is 57.\nThoughts: Since we need to use all four numbers to obtain 24, we can try to manipulate the numbers in a way that the sum of the resulting equation is close to', 'Observation: We can use the multiplication operation to combine the even numbers and create an odd number.\nThoughts: If we multiply 20 and 2, we get 40, which is an even number. However, if we add 1']
thoughts: ['Observation: The sum of the four given numbers is 57.\nThoughts: Since we need to use all four numbers to obtain 24, we can try to manipulate the numbers in a way that the sum of the resulting equation is close to', 'Observation: We can use the multiplication operation to combine the even numbers and create an odd number.\nThoughts: If we multiply 20 and 2, we get 40, which is an even number. However, if we add 1', 'Observation: We can combine the two 20s to get a larger even number.\nThoughts: We can add or multiply the two 20s to get an even number that can be combined with the other even number to make an odd number']
thoughts: ['Observation: The sum of the four given numbers is 57.\nThoughts: Since we need to use all four numbers to obtain 24, we can try to manipulate the numbers in a way that the sum of the resulting equation is close to', 'Observation: We can use the multiplication operation to combine the even numbers and create an odd number.\nThoughts: If we multiply 20 and 2, we get 40, which is an even number. However, if we add 1', 'Observation: We can combine the two 20s to get a larger even number.\nThoughts: We can add or multiply the two 20s to get an even number that can be combined with the other even number to make an odd number', 'Observation: We can combine the two 20s to get 40, which is an even number.\nThoughts: If we subtract 2 from 40, we get 38, which is an even number. We can then add ']
thoughts: ['Observation: The sum of the four given numbers is 57.\nThoughts: Since we need to use all four numbers to obtain 24, we can try to manipulate the numbers in a way that the sum of the resulting equation is close to', 'Observation: We can use the multiplication operation to combine the even numbers and create an odd number.\nThoughts: If we multiply 20 and 2, we get 40, which is an even number. However, if we add 1', 'Observation: We can combine the two 20s to get a larger even number.\nThoughts: We can add or multiply the two 20s to get an even number that can be combined with the other even number to make an odd number', 'Observation: We can combine the two 20s to get 40, which is an even number.\nThoughts: If we subtract 2 from 40, we get 38, which is an even number. We can then add ', 'Observation: We can use multiplication to combine the even numbers and get an odd number.\nThoughts: We can multiply 20 and 2 to get 40 and then subtract 20 from it to get 20. We can then add ']
Generated thoughts: ['Observation: The sum of the four given numbers is 57.\nThoughts: Since we need to use all four numbers to obtain 24, we can try to manipulate the numbers in a way that the sum of the resulting equation is close to', 'Observation: We can use the multiplication operation to combine the even numbers and create an odd number.\nThoughts: If we multiply 20 and 2, we get 40, which is an even number. However, if we add 1', 'Observation: We can combine the two 20s to get a larger even number.\nThoughts: We can add or multiply the two 20s to get an even number that can be combined with the other even number to make an odd number', 'Observation: We can combine the two 20s to get 40, which is an even number.\nThoughts: If we subtract 2 from 40, we get 38, which is an even number. We can then add ', 'Observation: We can use multiplication to combine the even numbers and get an odd number.\nThoughts: We can multiply 20 and 2 to get 40 and then subtract 20 from it to get 20. We can then add ']
We receive a state of type <class 'str'> For state:  Observation: The sum of the four given numbers is 57.
Thoughts: Since we need to use all four numbers to obtain 24, we can try to manipulate the numbers in a way that the sum of the resulting equation is close to 

state: 0.25
value: 0.25
2023-05-28 13:44:54,204 - INFO - State: Observation: The sum of the four given numbers is 57.
Thoughts: Since we need to use all four numbers to obtain 24, we can try to manipulate the numbers in a way that the sum of the resulting equation is close to, Value: 0.25
We receive a state of type <class 'str'> For state:  Observation: We can combine the two 20s to get 40, which is an even number.
Thoughts: If we subtract 2 from 40, we get 38, which is an even number. We can then add  

state: 0.25
value: 0.25
2023-05-28 13:44:55,242 - INFO - State: Observation: We can combine the two 20s to get 40, which is an even number.
Thoughts: If we subtract 2 from 40, we get 38, which is an even number. We can then add , Value: 0.25
We receive a state of type <class 'str'> For state:  Observation: We can combine the two 20s to get a larger even number.
Thoughts: We can add or multiply the two 20s to get an even number that can be combined with the other even number to make an odd number 

state: 0.42
value: 0.42
2023-05-28 13:44:56,282 - INFO - State: Observation: We can combine the two 20s to get a larger even number.
Thoughts: We can add or multiply the two 20s to get an even number that can be combined with the other even number to make an odd number, Value: 0.42
We receive a state of type <class 'str'> For state:  Observation: We can use multiplication to combine the even numbers and get an odd number.
Thoughts: We can multiply 20 and 2 to get 40 and then subtract 20 from it to get 20. We can then add  

state: 0.2
value: 0.2
2023-05-28 13:44:57,315 - INFO - State: Observation: We can use multiplication to combine the even numbers and get an odd number.
Thoughts: We can multiply 20 and 2 to get 40 and then subtract 20 from it to get 20. We can then add , Value: 0.2
We receive a state of type <class 'str'> For state:  Observation: We can use the multiplication operation to combine the even numbers and create an odd number.
Thoughts: If we multiply 20 and 2, we get 40, which is an even number. However, if we add 1 

state: 0.00625
value: 0.00625
2023-05-28 13:44:58,623 - INFO - State: Observation: We can use the multiplication operation to combine the even numbers and create an odd number.
Thoughts: If we multiply 20 and 2, we get 40, which is an even number. However, if we add 1, Value: 0.00625
We receive a state of type <class 'str'> For state:  Observation: The numbers given have a wide range of values and none of them are close to 24.
Thoughts: It might be helpful to break down the problem into smaller parts and find ways to manipulate the numbers to get closer to 24 

state: 0.75
value: 0.75
2023-05-28 13:44:59,850 - INFO - State: Observation: The numbers given have a wide range of values and none of them are close to 24.
Thoughts: It might be helpful to break down the problem into smaller parts and find ways to manipulate the numbers to get closer to 24, Value: 0.75
We receive a state of type <class 'tuple'> For state:  ('use 20,20,2,15  and basic arithmetic operations (+-*/) to obtain 24', 'Observation: The numbers given have a wide range of values and none of them are close to 24.\nThoughts: It might be helpful to break down the problem into smaller parts and find ways to manipulate the numbers to get closer to 24') 

Considering the thoughts you've had until now:

use 20,20,2,15  and basic arithmetic operations (+-*/) to obtain 24
Observation: The numbers given have a wide range of values and none of them are close to 24.
Thoughts: It might be helpful to break down the problem into smaller parts and find ways to manipulate the numbers to get closer to 24

Devise the next coherent thought that will aid in advancing the reasoning process and achieving a solution to use 20,20,2,15  and basic arithmetic operations (+-*/) to obtain 24. Assess various scenarios, think unconventionally, anticipate potential challenges, and resolve any outstanding queries. Tap into your mind's full potential and make certain no open questions remain.Write down your observations in format 'Observation:xxxx', then write down your thoughts in format 'Thoughts:xxxx'.
thoughts: ['Observation: The numbers given have a wide range of values and none of them are close to 24.\nThoughts: It might be helpful to break down the problem into smaller parts and find ways to manipulate the numbers to get closer to 24']
thoughts: ['Observation: The numbers given have a wide range of values and none of them are close to 24.\nThoughts: It might be helpful to break down the problem into smaller parts and find ways to manipulate the numbers to get closer to 24', 'Observation: Adding or subtracting 20 or 15 will result in a number too far from 24. Multiplying 20 and 15 will result in a number too large to work with. Dividing 20 or 15 by ']
thoughts: ['Observation: The numbers given have a wide range of values and none of them are close to 24.\nThoughts: It might be helpful to break down the problem into smaller parts and find ways to manipulate the numbers to get closer to 24', 'Observation: Adding or subtracting 20 or 15 will result in a number too far from 24. Multiplying 20 and 15 will result in a number too large to work with. Dividing 20 or 15 by ', 'Observation: One possible approach could be to use the numbers 20 and 2 to create a fraction or a decimal that is close to 1, and then use the number 15 to manipulate it to get closer to 24. \n\nThought']
thoughts: ['Observation: The numbers given have a wide range of values and none of them are close to 24.\nThoughts: It might be helpful to break down the problem into smaller parts and find ways to manipulate the numbers to get closer to 24', 'Observation: Adding or subtracting 20 or 15 will result in a number too far from 24. Multiplying 20 and 15 will result in a number too large to work with. Dividing 20 or 15 by ', 'Observation: One possible approach could be to use the numbers 20 and 2 to create a fraction or a decimal that is close to 1, and then use the number 15 to manipulate it to get closer to 24. \n\nThought', 'Observation: It might be helpful to use the fact that 20+20=40 and 15+2=17 to get closer to 24.\nThoughts: One possible approach is to use the numbers 20, 20, and']
thoughts: ['Observation: The numbers given have a wide range of values and none of them are close to 24.\nThoughts: It might be helpful to break down the problem into smaller parts and find ways to manipulate the numbers to get closer to 24', 'Observation: Adding or subtracting 20 or 15 will result in a number too far from 24. Multiplying 20 and 15 will result in a number too large to work with. Dividing 20 or 15 by ', 'Observation: One possible approach could be to use the numbers 20 and 2 to create a fraction or a decimal that is close to 1, and then use the number 15 to manipulate it to get closer to 24. \n\nThought', 'Observation: It might be helpful to use the fact that 20+20=40 and 15+2=17 to get closer to 24.\nThoughts: One possible approach is to use the numbers 20, 20, and', 'Observation: The number 2 seems to be the outlier in this set of numbers.\nThoughts: Instead of trying to use all four numbers, it might be more feasible to focus on manipulating the larger numbers (20 and 15) and using']
Generated thoughts: ['Observation: The numbers given have a wide range of values and none of them are close to 24.\nThoughts: It might be helpful to break down the problem into smaller parts and find ways to manipulate the numbers to get closer to 24', 'Observation: Adding or subtracting 20 or 15 will result in a number too far from 24. Multiplying 20 and 15 will result in a number too large to work with. Dividing 20 or 15 by ', 'Observation: One possible approach could be to use the numbers 20 and 2 to create a fraction or a decimal that is close to 1, and then use the number 15 to manipulate it to get closer to 24. \n\nThought', 'Observation: It might be helpful to use the fact that 20+20=40 and 15+2=17 to get closer to 24.\nThoughts: One possible approach is to use the numbers 20, 20, and', 'Observation: The number 2 seems to be the outlier in this set of numbers.\nThoughts: Instead of trying to use all four numbers, it might be more feasible to focus on manipulating the larger numbers (20 and 15) and using']
We receive a state of type <class 'str'> For state:  Observation: Adding or subtracting 20 or 15 will result in a number too far from 24. Multiplying 20 and 15 will result in a number too large to work with. Dividing 20 or 15 by  

state: 0.001
value: 0.001
2023-05-28 13:45:24,424 - INFO - State: Observation: Adding or subtracting 20 or 15 will result in a number too far from 24. Multiplying 20 and 15 will result in a number too large to work with. Dividing 20 or 15 by , Value: 0.001
We receive a state of type <class 'str'> For state:  Observation: It might be helpful to use the fact that 20+20=40 and 15+2=17 to get closer to 24.
Thoughts: One possible approach is to use the numbers 20, 20, and 

state: 0.16666666666666666
value: 0.16666666666666666
2023-05-28 13:45:25,963 - INFO - State: Observation: It might be helpful to use the fact that 20+20=40 and 15+2=17 to get closer to 24.
Thoughts: One possible approach is to use the numbers 20, 20, and, Value: 0.16666666666666666
We receive a state of type <class 'str'> For state:  Observation: One possible approach could be to use the numbers 20 and 2 to create a fraction or a decimal that is close to 1, and then use the number 15 to manipulate it to get closer to 24. 

Thought 

state: 0.4322916666666667
value: 0.4322916666666667
2023-05-28 13:45:27,501 - INFO - State: Observation: One possible approach could be to use the numbers 20 and 2 to create a fraction or a decimal that is close to 1, and then use the number 15 to manipulate it to get closer to 24. 

Thought, Value: 0.4322916666666667
We receive a state of type <class 'str'> For state:  Observation: The number 2 seems to be the outlier in this set of numbers.
Thoughts: Instead of trying to use all four numbers, it might be more feasible to focus on manipulating the larger numbers (20 and 15) and using 

state: 0.2
value: 0.2
2023-05-28 13:45:28,475 - INFO - State: Observation: The number 2 seems to be the outlier in this set of numbers.
Thoughts: Instead of trying to use all four numbers, it might be more feasible to focus on manipulating the larger numbers (20 and 15) and using, Value: 0.2
We receive a state of type <class 'str'> For state:  Observation: The numbers given have a wide range of values and none of them are close to 24.
Thoughts: It might be helpful to break down the problem into smaller parts and find ways to manipulate the numbers to get closer to 24 

state: 0.4
value: 0.4
2023-05-28 13:45:29,382 - INFO - State: Observation: The numbers given have a wide range of values and none of them are close to 24.
Thoughts: It might be helpful to break down the problem into smaller parts and find ways to manipulate the numbers to get closer to 24, Value: 0.4
We receive a state of type <class 'str'> For state:  Observation: The numbers given have a wide range of values and only one of them is odd.

Thoughts: Perhaps we can use the fact that only one of the numbers is odd to our advantage. We could try adding or subtracting the odd 

state: 0.4
value: 0.4
2023-05-28 13:45:31,493 - INFO - State: Observation: The numbers given have a wide range of values and only one of them is odd.

Thoughts: Perhaps we can use the fact that only one of the numbers is odd to our advantage. We could try adding or subtracting the odd, Value: 0.4
We receive a state of type <class 'str'> For state:  Observation: The numbers given have a wide range of values and only one of them is odd. 

Thoughts: Maybe I can use the fact that 15 is an odd number to my advantage. I could try subtracting it from one of 

state: 0.5
value: 0.5
2023-05-28 13:45:32,720 - INFO - State: Observation: The numbers given have a wide range of values and only one of them is odd. 

Thoughts: Maybe I can use the fact that 15 is an odd number to my advantage. I could try subtracting it from one of, Value: 0.5
output  []
2023-05-28 13:45:32,721 - ERROR - Error: max() arg is an empty sequence
2023-05-28 13:45:32,721 - INFO - Saving the current tree and metrics.
solution: None
kyegomez commented 1 year ago

Ah, the prompt is the issue then! We've implemented an plug in play command example in the readme.md!

Provide your own custom problem and let me know what you don't like about it