search

the-crypt-keeper / can-ai-code

Self-evaluating interview for AI coders
https://huggingface.co/spaces/mike-ravkine/can-ai-code-results
MIT License
513 stars 29 forks source link

Different prompt for Speechless CodeLlama v2 #150

Closed krzysiekpodk closed 6 months ago

krzysiekpodk commented 7 months ago

Apparently, correct prompt is Alpaca:

"You are an intelligent programming assistant.

Instruction:

Implement a linked list in C++

Response:"

Interestingly this was missing from model card so I have tested quite a few different prompts so far and it also did very good with Phind one.

I have not seen it produce code in multiple chunks but sometimes I add \n```python at the end of the prompt and it fixes the issue

Pros of this model is that its consitent in the thigs it gets right (and wrong as well), while Phind is very unsure about a lot of different questions. I'm thorn which of those two is still the best codellama

the-crypt-keeper commented 6 months ago

@krzysiekpodk thanks! almost perfect on junior-v2 and a decent showing on senior with this prompt.