TypeError while running harness.py

[mshihabr]

I am trying to run harness.py on swebench-lite. I got the following error:

Traceback (most recent call last):
  File "~/python3.11/site-packages/multiprocess/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "~/aider-swe-bench/harness.py", line 231, in process_one_instance
    coder = get_coder(
            ^^^^^^^^^^
  File "~/aider-swe-bench/harness.py", line 154, in get_coder
    coder = Coder.create(
            ^^^^^^^^^^^^^
  File "~/python3.11/site-packages/aider/coders/base_coder.py", line 138, in create
    res = coder(main_model, io, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Coder.__init__() got an unexpected keyword argument 'git_dname'

It seems like you are passing git_dname as a parameter to coder = Coder.create() but according to the latest code on aider-chat it does not have any parameter like this.

After commenting out that parameter, got another error TypeError: Coder.__init__() got an unexpected keyword argument 'max_chat_history_tokens'

[paul-gauthier]

Thanks for your interest in the aider SWE Bench harness.

Unfortunately it is not compatible with the latest version of aider. I will certainly update it eventually, but it's not likely to be an urgent priority at the moment.

[mshihabr]

No prob, commenting those two parameters seem to work

[magaton]

Hello, I am also trying to run harness.py with the current aider version (checkout from main) and hit the same problem. Unfortunately, commenting those 2 parameters does not solve the problem.

Namely:

message += problem_statement
                try:
                    coder.run(message)
                except Exception as coder_err:
                    # swallow any exceptions during benchmarking
                    dump(coder_err)
                    continue

always catches coder_err: coder_err: unsupported operand type(s) for +: 'function' and 'str'

I have tried to locate the Coder version where this constructor is still in place:

oder = Coder.create(
        main_model=model,
        io=io,
        #git_dname=git_dname,
        map_tokens=2048,  # Use 2k tokens for the repo map
        stream=False,
        auto_commits=False,  # Don't bother git committing changes
        fnames=oracle_files,
        auto_test=True,  # Automatically run the test_cmd after making changes
        test_cmd=test_cmd,
        # verbose=True,
        # edit_format="udiff",
        #max_chat_history_tokens=8*1024,
    )

but I could not. What version of Aider has been used for benchmarking?

@paul-gauthier I understand it is not your priority to fix this code, but do you have any suggestions on how to run it locally?

Thanks!

[jalammar]

@magaton PR #6 might be helpful for you