mrigankpawagi / HinglishEval

Evaluating the Effectiveness of Code-generation Models on Hinglish Prompts
1 stars 0 forks source link

Humaneval 019 Testcase need more thought on translation #13

Closed PrathamGupta423 closed 8 months ago

PrathamGupta423 commented 10 months ago

image

019 test is mostly english and english language based function. please suggest approch to translating this test.

mrigankpawagi commented 10 months ago

"delimited" is not a standard word and can be translated to Hindi. So we can say something like

numbers ek string hai jismei 'zero' se 'nine' tak ke numbers hai jinko spaces se alag kiya hua hai. [and so on]

@AnirudhG07 this is a good instance for including in the few-shot examples.

AnirudhG07 commented 10 months ago

from typing import List

def sort_numbers(numbers: str) -> str: """ Input ek space-delimited string hai jisme numbers 'zero' se 'nine' tak hote hai. Valid choices hai 'zero', 'one', 'two', 'three', 'four', 'five', 'six', 'seven', 'eight' aur 'nine'. Chhote se bade tak numbers ko sort karke string return karo.

sort_numbers('three one five') 'one three five' """

For this function, my main.py translated it as this in Hinglish. @mrigankpawagi is this fine?

mrigankpawagi commented 10 months ago

Feel free to close.