Missing 'Error Feedback' in AgentCoder Pipeline

[SmallGreenTurtle0-0]

Hello,

I've read your paper about AgentCoder. I notice that there is Error Feedback branch in your pipeline. However, I couldn't locate it in the provided code. Could you clarify or assist me with this?

[huangd1999]

Sorry for the late response as we are preparing our other works for the recent ddl.

For Error Feedback, we are used in https://github.com/huangd1999/AgentCoder/tree/5ef7a567044b410297b860642a74858a9163c045. I will develop AgentCoder in next month based on our newly works, which are not same as the current pipeline (also the technique report will be rewrite.

[SmallGreenTurtle0-0]

@huangd1999 Thank you, I`ve found the "Error Feedback".

However, in the code line, It looks like you import libraries that does not exist in requirements nor code.

[huangd1999]

Hi, did you mean (not sure) that the datasets library does not exist in the requirements? I will create a new requirements file based on the CodeGeeX. If you mean that the source code does not use the library, it is used in the next line in your reference.

[SmallGreenTurtle0-0]

@huangd1999 Oh, I mean I can not run python programmer_humaneval.py

Error is

Traceback (most recent call last): File "/AgentCoder_origin/programmer_humaneval.py", line 105, in from datasets import load_dataset ModuleNotFoundError: No module named 'datasets'

I think that you may miss some piece of code.

[huangd1999]

Got it. The key reason for this error is that we need first run pip install datasets.

[SmallGreenTurtle0-0]

Thank you so much.

After installing, 2 libraries:

openai==0.27.6
datasets==3.0.1

Now, I can run it.

[huangd1999]

Following the instruction of openai, I think it would be better if we can install pip install openai==0.28.0.

[SmallGreenTurtle0-0]

@huangd1999 I see you are using GPT-3.5-turbo. I wonder which gpt-3.5-turbo version have you used? (0301, 1106, 0613 or 0125).

[huangd1999]

Hi, In the previous experiments, we use gpt-3.5-turbo for the inference. It is not sure what version used in the experiments. But I suggest you also directly use gpt-3.5-turbo as some of the versions may not be accessible in the future.

[Shuyangg]

AgentCoder

Hi, did you finish the new development of AgentCoder? I'm confused by the KeyError: 'completion'. And I really want to use it on my own code problems.

[huangd1999]

Hi, the revised code is not fully finished yet. But we have completed the version that can be executed. Can you send an email to my address (hdong4948@gmail.com)? I can send you the revised version.

[huangd1999]

Dear Shuyang, I have send the zip into your email.