Support for Claude 3 - Githubissues

Codium-ai / AlphaCodium

Official implementation for the paper: "Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering""

https://www.codium.ai

GNU Affero General Public License v3.0

3.61k stars 270 forks source link

Support for Claude 3 #42

Closed gavrielc closed 5 months ago

gavrielc commented 7 months ago

Can AlphaCodium run on Claude 3 Opus?

It would be great to see how AlphaCodium using Claude 3 performs compared to AlphaCodium using GPT-4

mrT23 commented 5 months ago

done https://github.com/Codium-ai/AlphaCodium?tab=readme-ov-file#news-2024-17-05

gavrielc commented 5 months ago

@mrT23 This is amazing! I think many people will be interested in seeing how the models compare on AlphaCodium.

Can you add data labels to the leaderboard chart so that it's possible to easily see the exact percentage for each model?

mrT23 commented 5 months ago

@gavrielc good idea, added

To be honest, i was disappointed from Claude3. I thought it would do better. and given the fact it is slower and more expensive than GPT4, I don't think Anthropic are quite there yet in terms of competition for code models.