Open jwmatthews opened 2 months ago
I downloaded the full console output from "LOGLEVEL=debug make run-server" and then searched for the unique ways 'Updated File' is mentioned:
$ cat ./2024_June-24_kai_console_output.txt | grep "Updated" | sort -u
# Updated File
## Updated File
**Updated File**
I assume the root issue is that https://github.com/konveyor-ecosystem/kai/blob/main/kai/models/file_solution.py assume we will consistently see the Updated File represented in a section of "## Updated File
"
I have seen this too with Mixtral very often. As a workaround, I added following to the prompt:
// rest of the original prompt
{% if model_provider.model_id == 'mistralai/mixtral-8x7b-instruct-v01' %}
Only use ## to begin section headers in the markdown above.
{% endif %}
// llama header line
Using model:
mistralai/mixtral-8x7b-instruct-v01
from IBM BAMI am running example/run_demo.py
run_demo.py attempts to migrate 26 files, of this I see 6 failures, as in after exhausting all retries I see 6 files with messages of:
[pom.xml] Received status code 500
These are the 6 files where Kai is saying the LLM did not return an updated file,
This is reported as:
yet as I look at the debug output I can see the LLM did return an updated file.
Example of LLM Result info for CartEndPoint.java Example of LLM Result info for pom.xml
To aid debugging I captured the full console output in 2 separate gists below:
LOGLEVEL=debug make run-server
- Full output in gistExample of
kai/config.toml