search

cyberkaida / reverse-engineering-assistant

An AI assistant for reverse engineering tasks 👩‍💻
Apache License 2.0
165 stars 15 forks source link

Failed query engine #29

Closed velteyn closed 1 month ago

velteyn commented 4 months ago

Hello I started the project but after a simple question the program fails. Probably I'm giving the wrong question .. Dunno. Is my project setup wrong ? Thank you

immagine immagine immagine

velteyn commented 4 months ago

immagine

I tried to continue

velteyn commented 4 months ago

but after this the program stops again immagine

velteyn commented 4 months ago

ReVa-20240217-103831.reva.log ReVa-20240217-103831.reva.chat.txt Full chat log

cyberkaida commented 4 months ago

Thank you for trying the project!

The exception is because the model did not follow the schema. I have found that models below 70b parameters have trouble following the schema.

When running locally I am using mixtral:instruct with ollama. It is not as good as OpenAI, but it will follow the schema most of the time.

If you would like, you can try a function tuned model, though I have not tried this yet.

cyberkaida commented 4 months ago

I will try to catch this exception and pass it to the model so it can retry, I have found this works sometimes.

If this fails I will try to catch the exception and display a less scary message. Often just retrying can work (depending on temperature value) and a message should tell the user to try again.

I have been thinking about a dual model setup to allow for smaller models. One for the reasoning chain and one for the function calling.