Md-Ashraful-Pramanik / MapCoder

MapCoder: Multi-Agent Code Generation for Competitive Problem Solving
MIT License
67 stars 14 forks source link

Error while running on humaneval #6

Open breakices opened 1 month ago

breakices commented 1 month ago

Thanks for your awesome work. However, I met a error when running "python src/main.py --model ChatGPT --dataset HumanEval --strategy MapCoder"

Traceback (most recent call last):

File "C:\Users\breakices\Desktop\multi-agents-code\MapCoder-master\MapCoder-master\src\promptings\MapCoder.py", line 247, in run_single_pass for example_no, example in enumerate(response["problem"], start=1):

KeyError: 'problem'

I use your dataset and didn't change the code by the way. Thanks for your patient and open-source! Looking for your reply! @Md-Ashraful-Pramanik

breakices commented 1 month ago

I notice that the parse_xml() may return a dict like {'root': {'problem':['description':]},'algorithm':{}}, instead of {'problem':['description':],'algorithm':{}}

Md-Ashraful-Pramanik commented 1 month ago

The mapcoder algorithm needs the following xml structure

<root>
    <problem>
        <description>
        </description>

        <code>
        </code>

        <planning>
        </planning>

    </problem>

    <algorithm>
    </algorithm>
</root>

Otherwise it will raise an exception. In that case re-run it. I think ChatGPT gave wrong response that results in problem.

breakices commented 1 month ago

Thank you for your rapid reply! I will check the response and look for some solutions.