Further Llama Compatibility

[dibrale]

Extensive string parsing and prompt engineering changes to improve operation with llama LLMs. Included some new lines that can be uncommented for debugging. Changed max_tokens to 2000 for openai_call instances. Added escape mechanism from main loop.

[jmtatsch]

ast does not seem to be used.

For me this pr always stops after the first iteration.

babyagi    | *****USING LLAMA.CPP. POTENTIALLY SLOW.*****
babyagi    | 
babyagi    | *****OBJECTIVE*****
babyagi    | 
babyagi    | Solve world hunger
babyagi    | 
babyagi    | Initial task: Develop a task list
babyagi    | 
babyagi    | *****TASK LIST*****
babyagi    | 
babyagi    |  • Develop a task list
babyagi    | 
babyagi    | *****NEXT TASK*****
babyagi    | 
babyagi    | Develop a task list
babyagi    | 
babyagi    | llama_print_timings:        load time =  1090.65 ms
babyagi    | llama_print_timings:      sample time =     0.00 ms /     1 runs   (    0.00 ms per run)
babyagi    | llama_print_timings: prompt eval time =  1090.55 ms /     6 tokens (  181.76 ms per token)
babyagi    | llama_print_timings:        eval time =     0.00 ms /     1 runs   (    0.00 ms per run)
babyagi    | llama_print_timings:       total time =  1090.82 ms
babyagi    | 
babyagi    | llama_print_timings:        load time =  1351.25 ms
babyagi    | llama_print_timings:      sample time =    57.59 ms /   128 runs   (    0.45 ms per run)
babyagi    | llama_print_timings: prompt eval time =  6520.57 ms /    40 tokens (  163.01 ms per token)
babyagi    | llama_print_timings:        eval time = 42575.49 ms /   127 runs   (  335.24 ms per run)
babyagi    | llama_print_timings:       total time = 49169.21 ms
babyagi    | 
babyagi    | *****TASK RESULT*****
babyagi    | 
babyagi    | Task List for Solving World Hunger
babyagi    | 
babyagi    | 1. Conduct research to identify the root causes of world hunger.
babyagi    | 2. Develop and implement sustainable agriculture practices to increase food production.
babyagi    | 3. Create a network of food banks and distribution centers to ensure that food reaches those in need.
babyagi    | 4. Educate communities on proper nutrition and health practices to improve overall well-being.
babyagi    | 5. Collaborate with international organizations and governments to provide resources and support for hunger relief efforts.
babyagi    | 6. Encourage sustainable farming techniques, such as permac
babyagi    | 
babyagi    | llama_print_timings:        load time =  1090.65 ms
babyagi    | llama_print_timings:      sample time =     0.00 ms /     1 runs   (    0.00 ms per run)
babyagi    | llama_print_timings: prompt eval time = 20888.91 ms /   128 tokens (  163.19 ms per token)
babyagi    | llama_print_timings:        eval time =     0.00 ms /     1 runs   (    0.00 ms per run)
babyagi    | llama_print_timings:       total time = 20890.05 ms
babyagi    | 
babyagi    | llama_print_timings:        load time =  1351.25 ms
babyagi    | llama_print_timings:      sample time =    57.27 ms /   128 runs   (    0.45 ms per run)
babyagi    | llama_print_timings: prompt eval time = 37762.59 ms /   231 tokens (  163.47 ms per token)
babyagi    | llama_print_timings:        eval time = 43736.48 ms /   127 runs   (  344.38 ms per run)
babyagi    | llama_print_timings:       total time = 81572.84 ms
babyagi    | 
babyagi    | llama_print_timings:        load time =  1351.25 ms
babyagi    | llama_print_timings:      sample time =    56.62 ms /   128 runs   (    0.44 ms per run)
babyagi    | llama_print_timings: prompt eval time = 47153.82 ms /   287 tokens (  164.30 ms per token)
babyagi    | llama_print_timings:        eval time = 44042.29 ms /   127 runs   (  346.79 ms per run)
babyagi    | llama_print_timings:       total time = 91269.40 ms
babyagi    | Done.

[dibrale]

@jmtatsch - Thank you for testing my PR! The main loop completing means that tasks_storage was empty at the beginning of the subsequent iteration, which should not be happening given that a valid task list was returned. I will have a look at the code to ensure that task_storage is being filled correctly.

[francip]

Oops, I should've read more carefully the conversation. i merged this, but if there's any issues, let's get them fixed.

[dibrale]

Oops, I should've read more carefully the conversation. i merged this, but if there's any issues, let's get them fixed.

According to my latest testing, the issues seem to arise from Llama sometimes returning an empty response at the prioritization agent, which clears the task list and causes the program to terminate at the next while loop check. I already have a workaround for some of this, but I'm working on a more definitive solution.

[dibrale]

@francip - I submitted a pull request that should fix the issue @jmtatsch was having.