issues
search
princeton-nlp
/
SWE-bench
[ICLR 2024] SWE-Bench: Can Language Models Resolve Real-world Github Issues?
https://www.swebench.com
MIT License
1.47k
stars
241
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Fix: Allow to set GH token from env var in collect/print_pulls
#173
klieret
closed
2 days ago
2
matplotlib__matplotlib-18869 can't pass (?) due to test_tmpconfigdir_warning
#172
waterson
closed
4 days ago
1
Don't let tox install a virtualenv during evaluation
#171
waterson
closed
4 days ago
2
sphinx-doc instances create an unecessary tox virutalenv during eval
#170
waterson
closed
4 days ago
0
Handle failures because of None/empty patches
#169
klieret
closed
5 days ago
2
Update reporting and skip empty model patch predictions
#168
carlosejimenez
closed
5 days ago
1
Failing benchmark instances
#167
aorwall
opened
5 days ago
4
Fix newline outputs for django's log parser
#166
xingyaoww
closed
5 days ago
1
Passed test case count as failure?
#165
xingyaoww
closed
5 days ago
0
Fix so it doesn't crash when no env imgs to build
#164
JunShern
closed
5 days ago
1
Missing `validation.ipynb`?
#163
xingyaoww
closed
5 days ago
1
Fix evaluation hanging issue and improve patch apply
#162
xingyaoww
closed
6 days ago
2
Fix path to image in docs
#161
klieret
closed
6 days ago
0
`exec_run_with_timeout` does not actually kill long-running thread
#160
klieret
opened
1 week ago
1
Add timeout for overal execution of instance
#159
klieret
closed
1 week ago
1
swe-bench can get badly stuck in `future.result()`
#158
klieret
closed
6 days ago
2
docker evaluation gets stuck
#157
crhf
opened
1 week ago
3
Which Python version to use?
#156
anupamme
opened
1 week ago
2
Fix link to collection tutorial
#155
klieret
closed
6 days ago
0
It seems that current evaluation does not handle the apply failure case?
#154
Hodge931
opened
1 week ago
2
Various nitpicks
#153
klieret
closed
6 days ago
1
Add very simple CI
#152
klieret
closed
5 days ago
2
Fix: Support JSON datasets (avoid loading json twice)
#151
klieret
closed
6 days ago
0
Cannot load dataset from JSON file
#150
klieret
closed
6 days ago
0
Interface fix: run_id is required
#149
klieret
closed
6 days ago
0
Add a `schema_version: 2` field to evaluation output files
#148
klieret
opened
1 week ago
2
Missing metric/report.py
#147
donggrame
closed
5 days ago
2
inference part of project installs in plain packages list
#146
AnikinNN
opened
1 week ago
0
Get error "error: corrupt patch at line 40" when using the gold patch of "django__django-15202"
#145
BoxiYu
closed
5 days ago
4
Running into errors during evaluation
#144
ivan4722
opened
1 week ago
6
Can't test installation in setup, I get error
#143
ivan4722
closed
1 week ago
2
Containerize SWE-bench evaluation
#142
carlosejimenez
closed
1 week ago
1
Problem with conda install gxx_linux-64 gcc_linux-64 make -y on OSX
#141
ivan4722
closed
1 week ago
0
Test cases
#140
Hodge931
closed
2 weeks ago
1
Skipped test cases
#139
Hodge931
closed
2 weeks ago
1
Distinguish Between Verified and Unverified Solutions
#138
thisdotmatt
closed
2 weeks ago
3
Problem in sympy__sympy-13773
#137
Hodge931
closed
1 week ago
2
Reproducing the tests using run_evaluation.py
#136
nasr020
closed
1 week ago
1
Where can I find `swe-bench.json`?
#135
yorhaha
closed
3 weeks ago
2
Add pytest to pydicom requirements
#134
Danila89
closed
3 weeks ago
1
When and how should `hints_text` be used?
#133
atinylittleshell
closed
3 weeks ago
1
Inference and evaluate on SWE-Bench faster by reusing previous built env
#132
Yuzz1020
closed
1 month ago
1
Consider the presence of downstream information in `problem_statement`
#131
dustinbyrne
closed
1 month ago
0
Update astropy pre_install for only 4.0> versions
#130
carlosejimenez
closed
1 month ago
0
Fix astropy installation after setuptools updated to 70.0.0
#129
carlosejimenez
closed
1 month ago
0
Clarification Needed on Removal of Instances with Error Message Checks in SWE-bench Lite Dataset
#128
ramsey-coding
closed
2 weeks ago
2
Install failed on instances from astropy__astropy
#127
JiyangZhang
closed
1 month ago
3
`model_name_or_path` is None when running models without adapters, causing an error in `run_evaluation.py`
#126
rucnyz
closed
2 weeks ago
2
what's the difference between environment_setup_commit and base_commit?
#125
ramsey-coding
closed
1 month ago
6
how to download one task instance from SWE-bench dataset?
#124
ramsey-coding
closed
1 month ago
1
Next