issues
search
nuprl
/
MultiPL-E
A multi-programming language benchmark for LLMs
https://nuprl.github.io/MultiPL-E/
Other
206
stars
38
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Typescript translation of union-type arrays is incorrect
#160
nielstron
opened
1 week ago
2
error in execution
#159
BingyuanZhang
closed
1 week ago
1
Language Names in MBPP Prompts
#158
timothytmzhou
opened
1 month ago
1
separate containers per language
#157
arjunguha
opened
2 months ago
0
autochatmodel.py: a way to ignore prompts?
#156
arjunguha
closed
2 months ago
2
MultiPL-E framework does not seem to be able to produce valid completion results for Qwen model
#155
brick-pid
closed
2 months ago
2
New LeetCode dataset
#154
cassanof
closed
2 months ago
2
add support for translating Dart
#153
devoncarew
closed
2 months ago
15
Add support for the Dart language
#152
devoncarew
closed
2 months ago
3
Adding C language to MultiPL-E
#151
49Simon
closed
3 months ago
1
Evaluation on HumanEval for Python not possible anymore
#150
Devy99
closed
3 months ago
2
Update README.md
#149
arjunguha
closed
4 months ago
0
Translated Prompts In Hugging Face not complete
#148
49Simon
closed
4 months ago
3
Fix Elixir install
#147
mvkvc
closed
4 months ago
4
Does automodel_vllm need to produce logprobs
#146
arjunguha
opened
4 months ago
1
MultiPL-E 3.0 release
#145
arjunguha
closed
4 months ago
2
Run generations of multiple languages at once
#144
KevinZeng08
closed
4 months ago
1
Performance issue with Julia/R languages
#143
Devy99
closed
4 months ago
6
Task HumanEval/092 has contradictory tests in Rust
#142
geajack
opened
5 months ago
5
How to run a multi-GPU model for inference testing?
#141
huangmenglong
closed
6 months ago
2
Citation for the LeetCode Dataset
#140
JJGO
closed
6 months ago
1
Return logprobs for openai
#139
cassanof
closed
7 months ago
0
Add log probability information in the completions
#138
cassanof
closed
7 months ago
0
Quantized model is not supported - Calling cuda() is not supported for 4-bit or 8-bit quantized models
#137
Santhoshkumar-p
closed
7 months ago
2
Add Clojure programming language
#136
puredanger
closed
7 months ago
9
leetcode dataset not found
#134
jonastuttle
closed
6 months ago
1
v0.5 proposal
#133
cassanof
closed
7 months ago
2
Fixes for Coq and Lean
#132
cassanof
closed
8 months ago
0
The "automodel" file does not have this parameter.
#131
shuaiwang2022
closed
9 months ago
0
code generated with wrong end of string place
#128
tedvuminhhuy
closed
9 months ago
2
padding left some token causing compile error
#127
tedvuminhhuy
closed
9 months ago
2
Coq and Lean
#125
JakeGinesin
closed
8 months ago
3
Unable to load weights from pytorch checkpoint file
#124
LeVuMinhHuyWindows
closed
9 months ago
1
fix coerce case
#123
cassanof
closed
10 months ago
0
fix
#122
cassanof
closed
11 months ago
0
Dafny
#121
cassanof
closed
11 months ago
1
Evaluation with a container stops halfway without error message
#120
Chen-Hailin
closed
11 months ago
4
cap model len
#119
cassanof
closed
12 months ago
0
fix
#118
cassanof
closed
1 year ago
0
Add elixir
#117
mvkvc
closed
12 months ago
11
Set of minor fixes
#116
Randl
closed
1 year ago
1
Multiline comments regexp fix
#115
Randl
closed
1 year ago
1
All non-multiline commented prompts currently broken
#114
cassanof
closed
1 year ago
0
fix r
#113
cassanof
closed
1 year ago
0
add tokenizer revision to vllm
#112
cassanof
closed
1 year ago
0
R prompts are currently broken
#111
arjunguha
closed
1 year ago
0
bump memory
#110
cassanof
closed
1 year ago
0
don't print desc
#109
cassanof
closed
1 year ago
0
fix R canonical prompt translation
#108
cassanof
closed
1 year ago
0
Update automodel_vllm.py
#107
JohnGouwar
closed
1 year ago
0
Next