nuprl / MultiPL-E

A multi-programming language benchmark for LLMs
https://nuprl.github.io/MultiPL-E/
Other
201 stars 38 forks source link

Typed Luau translator #101

Closed cassanof closed 1 year ago

cassanof commented 1 year ago

I also wrote a small verification script that makes sure the translated types actually make sense (dataset_builder/verification/verify_luau_types.py).

The issue now is evaluation. What do we do? Do we enforce well-typedness (--!strict)?

cassanof commented 1 year ago

looks amazing!