search

Md-Ashraful-Pramanik / MapCoder

MapCoder: Multi-Agent Code Generation for Competitive Problem Solving
MIT License
57 stars 10 forks source link

The benchmark may be so low #7

Open lianqingWu opened 1 week ago

lianqingWu commented 1 week ago

the question is about benchmark, when i test no matter humaneval and so on, the chatgpt with gpt3 use "Direct" can solve at least 75%,

Md-Ashraful-Pramanik commented 1 week ago

Dear Tome, Due to the advancements of the chatgpt models you may get good results.

On Fri, Sep 20, 2024, 12:37 PM Tome @.***> wrote:

the question is about benchmark, when i test no matter humaneval and so on, the chatgpt with gpt3 use "Direct" can solve at least 75%,

— Reply to this email directly, view it on GitHub https://github.com/Md-Ashraful-Pramanik/MapCoder/issues/7, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANO3RTTVBOD3Y7STAOADMJ3ZXO7CNAVCNFSM6AAAAABORLWA5SVHI2DSMVQWIX3LMV43ASLTON2WKOZSGUZTQMBQGI3DSMA . You are receiving this because you are subscribed to this thread.Message ID: @.***>