search

meysamhadeli / codai

Codai is an AI code assistant that helps developers through a session-based CLI, providing intelligent code suggestions, refactoring, and code reviews based on the full context of their projects. It supports multiple LLMs, including GPT-4o, GPT-4 and Ollama, to streamline daily development tasks.
Apache License 2.0
23 stars 0 forks source link

Qualitative Comparison of Models? #61

Closed rljacobson closed 4 hours ago

rljacobson commented 5 hours ago

As someone who wants to try this out, my first question is, which LLM should I use? It's really cool that there are so many options, but surely some perform better than others. I understand there may be some subjectivity involved, but I would find even vague general guidance helpful.

I did read the readme, but it's possible that you have given such guidance and I've missed it, in which case never mind.

meysamhadeli commented 5 hours ago

As someone who wants to try this out, my first question is, which LLM should I use? It's really cool that there are so many options, but surely some perform better than others. I understand there may be some subjectivity involved, but I would find even vague general guidance helpful.

I did read the readme, but it's possible that you have given such guidance and I've missed it, in which case never mind.

Hi, In readme I mentioned that best model now is gpt-4o and for local model ollama like Phi-3-medium instruct (128k) and Mistral Large (2407) are good in the result but not very accurate like gpt-4o.