search

Aider-AI / refactor-benchmark

Aider's refactoring benchmark exercises based on popular python repos
Apache License 2.0
46 stars 2 forks source link

Running instruction? #4

Open jacklxc opened 1 week ago

jacklxc commented 1 week ago

Hi, is there any instruction to run this refactor benchmark?

paul-gauthier commented 1 week ago

Thanks for your interest in aider and the refactor benchmark.

You can run it like the main aider benchmark:

https://github.com/Aider-AI/aider/tree/main/benchmark

You just need to use --exercises-dir to point it at the refactor exercises.

jacklxc commented 1 week ago

Thanks for your quick reply. I was diving into the source code but still confused. Can you explain what is the outputs of the LLM and how is the output applied to the original file? Is the LLM output the full updated file content or only the difference between the original file content and the updated file content?

I am trying to use my own LLM for this benchmark, thank you.