nuprl / MultiPL-E

A multi-programming language benchmark for LLMs
https://nuprl.github.io/MultiPL-E/
Other
201 stars 38 forks source link

Small issues with Swift prompt signatures #63

Closed PootieT closed 1 year ago

PootieT commented 1 year ago

Issue 1

The indentation for extension is funky here

    def add_protocol_conformance(self, t: str, p: str, body: str):
        conf_str: str = f"""
extension {t}: {p} {body}
        """
        self.protocol_conformances.add(conf_str)

Instead, this should fix it:

    def add_protocol_conformance(self, t: str, p: str, body: str):
        conf_str: str = f"""
extension {t}: {p} {body}
"""
        self.protocol_conformances.add(conf_str)

the resulting extension and function will be nicely aligned afterwards at the base indentation level.

issue2

Not sure if this was intentional but there are certainly other default libraries imported for other languages, so maybe this is needed in the prompt:

import Swift
import Foundation

There are quite a few functions that uses pow, ceil, round, floor, or sqrt and they can all be found in the libraries above.

arjunguha commented 1 year ago

Would you be interested in submitting a PR?

There is this from a while ago: https://github.com/nuprl/MultiPL-E/pull/26

PootieT commented 1 year ago

will do EOD

PootieT commented 1 year ago

https://github.com/nuprl/MultiPL-E/pull/69