QuangBK
commented
1 year ago The model decides to give you [Action/Final Answer], we cannot control it. To get rid of this, there are two possible ways:
First option:
The prompt gives you the logprobs, so you can apply a threshold like this:
if result_mid['logprobs']['Final Answer'] > threshold:
return the FinalAnswerI will update the code with the threshold later.
Second option: Choose the better model, usually, more parameters give you better reasoning. I see fewer loop problem happening in 33B model (this happens more often in 7B models).
schauppi
Hello Bud - Thx for this great work!
I testet your code with the same vicuna model. If I run the agent I get the following output:
`### Input: What is the population of Congo?
Response:
Question: What is the population of Congo? Thought: First, I need to find the population of Congo. Action: Google Search Action Input: What is the population of Congo? Observation: 95.89 million (2021) Thought: Now, I should check the source of this information. Action: Google Search Action Input: Is 95.89 million the correct population of Congo? Observation: Democratic Republic of the Congo: Population size, in millions: For that indicator, ... minimum of 15.28 million in 1960 and a maximum of 95.89 million in 2021. From 1960 to 2021 the population of the Democratic Republic of the Congo increased from 15.25 million to 95.89 million people. This is a growth of 528.9 ... the DR Congo 2020 population is estimated at 89,561,403 people at mid year according to UN data. the DR Congo population is equivalent to 1.15% of the total ... Missing: 95.89 | Must include:95.89. Congo 2020 population is estimated at 5,518,087 people at mid year according to UN data. Congo population is equivalent to 0.07% of the total world population. Missing: 95.89 million. Population, Total. 95.89 million. (as of 2021) ; GNI, Atlas Method. 52.54 billion. (in current US$ as of 2021) ; GNI per capita, Atlas Method. 550. (in current US ... The Republic of Congo (Congo-Brazzaville), located in Central Africa, covers 342,000 km2. Its population of 5.7 million people is largely young, ... Missing: 95.89 | Must include:95.89. By land area, the DRC is the second-largest country in Africa, after Algeria, and the 11th-largest in the world.With a population of around 112 million, ... Country Characteristics ; Population, Total. 95.89 million. (as of 2021) ; GNI, Atlas Method. 52.54 billion. (in current US$ as of 2021) ; GNI per capita, Atlas ... With a current population of about 89.56 million, DR Congo is projected to surpass 100 million in 2024 and double its population by 2047. DR Congo has one of ... Missing: 95.89 | Must include:95.89. Thought: I now know the final answer. Action: Google Search Action Input: What is the population of Congo? Observation: 95.89 million (2021) Thought: I now know the final answer. Final Answer: The population of Congo is 95.89 million (2021).`
If I change the num_iter parameter to 5, the agent plays the game further. I cannot find the solution - maybe you have an answer? (The agents knows the final answer but continues with the search).
Edit: I See - you have the same "problem" in the gradio demo image. The model returns: "I know now the final answer" And the next action is "Google Search" instead "Final Answer:".