nuprl / MultiPL-E

A multi-programming language benchmark for LLMs
https://nuprl.github.io/MultiPL-E/
Other
201 stars 38 forks source link

PHP test indexed array comparison #72

Open PootieT opened 1 year ago

PootieT commented 1 year ago

example problem: HumanEval_26_remove_duplicates

function test(): void {
    print_r(candidate(array(1, 2, 3, 2, 4, 3, 5))); // this gives
//    Array
//(
//    [0] => 1
//    [4] => 4
//    [6] => 5
//)
    if (candidate(array(1, 2, 3, 2, 4, 3, 5)) !== array(1, 4, 5)) { throw new Exception("Test failed!"); }
}

sometimes, the built-in array processing functions (array_filter for example) in PHP keeps the original index of the array, make this test fail, but somehow I feel like this should be considered a pass? (again, this maybe have been debated before, but just throwing it out there)

A possible solution is what was discussed here

function array_equal($a, $b) {
    return (
         is_array($a) 
         && is_array($b) 
         && count($a) == count($b) 
         && array_diff($a, $b) === array_diff($b, $a)
    );
}

// and then for tests
if (!array_equal(candidate(array(1, 2, 3, 2, 4, 3, 5)),array(1, 4, 5))) {throw new Exception("Test failed!"); };