Run generations of multiple languages at once

nuprl / MultiPL-E

A multi-programming language benchmark for LLMs

https://nuprl.github.io/MultiPL-E/

Other

200 stars 38 forks source link

Run generations of multiple languages at once #144

Closed KevinZeng08 closed 4 months ago

KevinZeng08 commented 4 months ago

Hi, I am wondering whether this framework can run generations of multiple languages at once? It seems that every time I execute automodel.py, there is a must to pass --lang parameter.

arjunguha commented 4 months ago

It’s not something we support. It would be quite a change.