search

the-crypt-keeper / can-ai-code

Self-evaluating interview for AI coders
https://huggingface.co/spaces/mike-ravkine/can-ai-code-results
MIT License
513 stars 29 forks source link

Evaluate codellama/CodeLlama-*-Python using code-completion objective #159

Closed the-crypt-keeper closed 6 months ago

the-crypt-keeper commented 7 months ago

Just noticed that under 'Model capabilities' of the Python variants we do not expect these models to follow instructions, but only complete code.