Problem in run_repobench_r.py

Leolty / repobench

✨ RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems - ICLR 2024

https://arxiv.org/abs/2306.03091

Creative Commons Attribution 4.0 International

133 stars 7 forks source link

Problem in run_repobench_r.py #12

Open yhshu opened 6 months ago

yhshu commented 6 months ago

When I downloaded data and tried to run run_repobench_r.py, I got this:

Exception has occurred: KeyError
'test'
  File "/research/nfs_su_809/workspace/shu.251/repobench/run_repobench_r.py", line 22, in main
    data_first = data_first["test"]
  File "/research/nfs_su_809/workspace/shu.251/repobench/run_repobench_r.py", line 92, in <module>
    fire.Fire(main)
KeyError: 'test'

I only saw easy and hard in data_first. There is no 'test' in data_first. Could you provide me with some clues? Thanks.

Leolty commented 5 months ago

Sorry for the late response, just comment out those two lines should fix this:

data_first = data_first["test"]
data_random = data_random["test"]

Sorry this might be due to the difference versions of data...