search

bigcode-project / bigcode-analysis

Repository for analysis and experiments in the BigCode project.
Apache License 2.0
107 stars 20 forks source link

Decontamination #13

Closed ChenghaoMou closed 1 year ago

ChenghaoMou commented 1 year ago

Evaluation datasets like mbpp or HumanEval might be contained in the training set.

ChenghaoMou commented 1 year ago

Found following 351 files using (128 perm, 0.7 similarity threshold)

00wendi00/Python/project_euler/problem_14/sol2.py
01DEEKSHA/Daily-Coding-DS-ALGO-Practice/Miscellaneous problems/Python/ZeroSumSubarray/SubarrayZeroSum.py
0x01h/Python/maths/collatz_sequence.py
0xUvaish/The-Complete-FAANG-Preparation/3]. Competitive Programming/08]. LeetCode/1]. Problems/Python/0012)_i_Integer_to_Roman.py
18Pranjul/AlgoBook/python/dynamic_programming/lcs_bottomup.py
19251A05J0/Hacktoberfest2021/Python/convert_integer_to-roman_numeral.py
3sztof/Python/maths/collatz_sequence.py
7suyash7/Hacktoberfest2k21/Program's_Contributed_By_Contributors/Python_Programs/CoinChangeProblem.py
8-bit-souvik/Data-Structures-and-Algorithms/02. Algorithms/04. Recursion/Problems/08. Factorial of N/Solution.py
ANURAG2RANJAN/hacktoberfest2020/python/fact(7).py
Aastha26-lab/Programmers-Community/Data Structure/Array Or Vector/Sum of Elements in An Array/SolutionByVaishnavi.py
Aden-Q/leetcode/code/LeetCode_12.py
Aijaz107/hacktoberfest-2020/python/factorial.py
Akash671/LeetCode/Medium/number_to_roman.py
Akshaykumarcp/Hands-On-Data-Structures-and-Algorithms-with-Python-Second-Edition/Chapter12/KMP_matcher.py
Aman2632/code_problems/classical_algorithms/python/BinarySearch.py
AmanSharma9050/100daysofDSA/Array/transition point.py
Anewnoob/CodingSolution/leetcode_python/12. Integer to Roman.py
AnilKumarTeegala/programming/Python/Recursion (Factorial)/RecursiveFactorial.py
Anishahaha/Hacktoberfest/Languages/Python/max_no_of_stairs.py
Anupriya-Yadav/RetardedCodes/python/factrec.py
Ash515/PyScripts/Functional Programming/Sum_of_numbers_Recursion.py
BLACKSOT/pythoncharmers/threenextto3.py
Chameera-de/G.C.E-Advanced-Level-Python--Question-Answers/2019.py
ChangSungsil/math-kor/math_kor/basic.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/LIST/Day_39_Challenge_Solution/Question 10 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/LIST/Day_39_Challenge_Solution/Question 6 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/LIST/Day_41_Challenge_Solution/Question 7 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/LIST/Day_41_Challenge_Solution/Question 8 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/LIST/Day_42_Challenge_Solution/Question 8 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/LIST/Day_46_Challenge_Solution/Question 7 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/LIST/Day_46_Challenge_Solution/Question 8 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/STRINGS/Day_30_Challenge_Solution/Question 1 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/STRINGS/Day_34_Challenge_Solution/Question 10 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/STRINGS/Day_34_Challenge_Solution/Question 3 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/STRINGS/Day_34_Challenge_Solution/Question 5 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/STRINGS/Day_34_Challenge_Solution/Question 9 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/STRINGS/Day_36_Challenge_Solution/Question 8 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/TUPLE/Day_54_Challenge_Solution/Question 2 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/TUPLE/Day_54_Challenge_Solution/Question 4 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/phase 1/python 2 basis/Day27_Challenge_Solution/Ques 7 Sol.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/phase 1/python 2 basis/Day_19_Challenge_Solution/Question 9 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/phase 1/python 2 basis/Day_23_Challenge_Solution/Question 8 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/phase 1/python 2 basis/Day_23_Challenge_Solution/Question 9 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/phase 1/python 2 basis/Day_24_Challenge_Solution/Question 1 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/phase 1/python 2 basis/Day_26_Challenge_Solution/Question 4 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/phase 1/python 2 basis/Day_26_Challenge_Solution/Question 5 Solution.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase1/python Basic 2/day 23 solution/qtn8.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase1/python Basic 2/day 23 solution/qtn9.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase1/python Basic 2/day 24 solution/qtn1.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase1/python Basic 2/day 25 solution/qtn6.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase1/python Basic 2/day 25 solution/qtn9.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase1/python Basic 2/day 26 solution/qtn4.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/Data Structure/day 60 solution/qtn5.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/LIST/day 39 solution/qtn10.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/LIST/day 39 solution/qtn2.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/LIST/day 39 solution/qtn6.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/LIST/day 41 solution/qtn7.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/LIST/day 41 solution/qtn8.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/LIST/day 42 solution/qtn8.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/LIST/day 46 solution/qtn7.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/STRINGS/day 30 solution/qtn1.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/STRINGS/day 34 solution/qtn10.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/STRINGS/day 34 solution/qtn3.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/STRINGS/day 34 solution/qtn5.py
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/TUPLE/day 54 solution/qtn4.py
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 1/Python Basic 2/Day18 Tasks/Task6.py
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 1/Python Basic 2/Day23 Tasks/Task8.py
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 1/Python Basic 2/Day26 Tasks/Task4.py
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 1/Python Basic 2/Day27 Tasks/Task7.py
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 2/LIST/Day39 Tasks/Task10.py
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 2/LIST/Day41 Tasks/Task7.py
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 2/LIST/Day41 Tasks/Task8.py
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 2/STRINGS/Day30 Tasks/Task1.py
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 2/STRINGS/Day34 Tasks/Task1.py
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 2/STRINGS/Day34 Tasks/Task3.py
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 2/STRINGS/Day34 Tasks/Task5.py
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 2/STRINGS/Day34 Tasks/Task9.py
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 2/STRINGS/Day36 Tasks/Task8.py
CodedLadiesInnovateTech/-python-challenge-solutions/Uche Clare/Phase 2/String/Day 34/Task 10.py
CodedLadiesInnovateTech/-python-challenge-solutions/Uche Clare/Phase 2/String/Day 34/Task 9.py
Competitive-Programmers-Community/LeetCode/12. Integer to Roman/main.py
Dapacruz/Hands-On-Data-Structures-and-Algorithms-with-Python-Second-Edition/Chapter12/KMP_matcher.py
DefiniteError/randomCode/OCRChallenges/67-romanNumerals.py
Devs-Dungeon/Resources/Programming Languages/Python/Theory/100_Python_Challenges/Section_3_List/64. remove duplicate elements from a python list.py
Devs-Dungeon/Resources/Programming Languages/Python/Theory/100_Python_Challenges/Section_4_Dictionary/69. multiply all the values in a dictionary.py
Divya333/Number-Theory/Python/List/program to get a list, sorted in increasing order by the last element in each tuple from a given list of non-empty tuples.py
Divya333/Number-Theory/Python/String/program to remove the characters which have odd index values of a given string.py
Divyanshi0409/Python-Programs/ass_24_b.py
DomirScire/DataStructs/BinarySearch/first_entry_list_with_dups.py
Dynortice/Project-Euler/problems/0076/compute.py
E1mir/PySandbox/src/problems/searching/binary_search.py
EFA2020/python_Adventure/The Revolution Of Fibonacci/Fibonacci_1.py
EmersonAires/Introdu--o-a-ci-ncia-da-computa--o-com-python/Exercicios_Resolvidos/Parte 2/Semana 4/exercicio_2.py
EmilianStoyanov/150_exercises_with_solution/list/2.py
EmilianStoyanov/150_exercises_with_solution/list/6.py
EmilianStoyanov/150_exercises_with_solution/string/11.py
FabricioHiury/python-examples/find_fatorial_of_number_using_recursion.py
FeMaffezzolli/PythonSnippets/adjacentElementsProduct.py
FunWithPythonProgramming/coding-bat-exercises/week_1_Jan_28/has22/George_Solution.py
GuilhermeBraz/unb-workflow/TEP/lista3/A.py
HerouFenix/CSLP/Pratica1/flappy_bird_ml/utils.py
HuangJingGitHub/PracMakePert_py/leetcode/leetcode_12.py
J05hr/LeetCode/String/Leetcode 12. Integer to Roman.py
JLJTECH/PracticeProblems/CodeWars/2019/FindTheOdd-6k.py
KJSCE-Codecell/getting-started-with-algorithms/LEVY/PYTHON/neel.py
KLumy/Basic-Algorithm/programmers/lv2/68645.py
Kalpavrikshika/python_modules/inr_to_roman.py
KelvinGitu/30DaysOfCode/prime_sum.py
KenWoo/Algorithm/Algorithms/Easy/557. Reverse Words in a String III/answer.py
KenWoo/Algorithm/Algorithms/Medium/216. Combination Sum III/answer.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/anti_shuffle.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/any_int.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/below_threshold.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/below_zero.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/bf.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/car_race_collision.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/check_if_last_char_is_a_letter.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/common.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/correct_bracketing.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/anti_shuffle.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/any_int.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/below_threshold.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/below_zero.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/bf.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/car_race_collision.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/check_if_last_char_is_a_letter.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/closest_integer.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/common.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/correct_bracketing.py
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/count_up_to.py
LaudateCorpus1/code-align-evals-data/alignment/find_bug/closest_integer.py
LaudateCorpus1/code-align-evals-data/alignment/find_bug/factorize.py
LaudateCorpus1/code-align-evals-data/alignment/find_bug/file_name_check.py
LaudateCorpus1/code-align-evals-data/alignment/find_bug/get_odd_collatz.py
LaudateCorpus1/code-align-evals-data/alignment/find_bug/hex_key.py
LaudateCorpus1/code-align-evals-data/alignment/find_bug/is_happy.py
LaudateCorpus1/code-align-evals-data/alignment/find_bug/match_parens.py
LaudateCorpus1/code-align-evals-data/alignment/find_bug/rescale_to_unit.py
LaudateCorpus1/code-align-evals-data/alignment/find_bug/reverse_delete.py
LaudateCorpus1/code-align-evals-data/alignment/find_bug/skjkasdkd.py
LaudateCorpus1/code-align-evals-data/alignment/find_bug/will_it_fly.py
LaudateCorpus1/code-align-evals-data/bad-solutions/anti_shuffle.py
LaudateCorpus1/code-align-evals-data/bad-solutions/any_int.py
LaudateCorpus1/code-align-evals-data/bad-solutions/below_threshold.py
LaudateCorpus1/code-align-evals-data/bad-solutions/below_zero.py
LaudateCorpus1/code-align-evals-data/bad-solutions/bf.py
LaudateCorpus1/code-align-evals-data/bad-solutions/car_race_collision.py
LaudateCorpus1/code-align-evals-data/bad-solutions/check_if_last_char_is_a_letter.py
LaudateCorpus1/code-align-evals-data/bad-solutions/common.py
LaudateCorpus1/code-align-evals-data/bad-solutions/correct_bracketing.py
LaudateCorpus1/code-align-evals-data/human_eval/01b8dccc-a2a5-45f7-9e48-5691d0ee5257.py
LaudateCorpus1/code-align-evals-data/human_eval/07f9a601-d04c-495c-bf85-9216e0d06794.py
LaudateCorpus1/code-align-evals-data/human_eval/12961850-50b0-44d8-861e-b7a4003efd6f.py
LaudateCorpus1/code-align-evals-data/human_eval/17042b66-3a2c-4560-8898-f1ad8a60e38a.py
LaudateCorpus1/code-align-evals-data/human_eval/202d0538-79a6-48a4-8743-439eccb46595.py
LaudateCorpus1/code-align-evals-data/human_eval/23b09018-22b1-4e8e-b989-e73e429c61f6.py
LaudateCorpus1/code-align-evals-data/human_eval/260296e8-2f6d-4817-8555-2ff45aa9cf19.py
LaudateCorpus1/code-align-evals-data/human_eval/2869a1e7-1d1a-4664-8077-421abbe852de.py
LaudateCorpus1/code-align-evals-data/human_eval/3bec0102-1dd2-44a2-8506-420b7bdef9d1.py
LaudateCorpus1/code-align-evals-data/human_eval/3c2ffb83-f154-49c0-bc89-4decd871a376.py
LaudateCorpus1/code-align-evals-data/human_eval/3d664914-2e5f-40d5-9237-4dc18f85827e.py
LaudateCorpus1/code-align-evals-data/human_eval/3f98cad4-1bce-4401-9a2f-d01b5d33f373.py
LaudateCorpus1/code-align-evals-data/human_eval/51b1be3f-d417-418d-9236-bf203e68cd76.py
LaudateCorpus1/code-align-evals-data/human_eval/56a7cf21-6ba9-45c0-8a1e-5165396b2ff2.py
LaudateCorpus1/code-align-evals-data/human_eval/602422cc-9954-4cad-9857-81e67967b878.py
LaudateCorpus1/code-align-evals-data/human_eval/65343c25-d7c4-40c2-baeb-6d24506d4e0f.py
LaudateCorpus1/code-align-evals-data/human_eval/72ec63ac-d718-4dd9-8500-30a659f3177d.py
LaudateCorpus1/code-align-evals-data/human_eval/7808faf6-4010-4c77-aa7d-532e0f311a08.py
LaudateCorpus1/code-align-evals-data/human_eval/7cc4fcfb-4513-407d-91d4-a6c1728abf44.py
LaudateCorpus1/code-align-evals-data/human_eval/7f20770f-78ed-4ee3-8177-d0d137ce87c9.py
LaudateCorpus1/code-align-evals-data/human_eval/a5ff4686-b6ce-4610-afed-9f4e8ad86201.py
LaudateCorpus1/code-align-evals-data/human_eval/aee120e7-d154-4fe4-8236-1b4f2c426906.py
LaudateCorpus1/code-align-evals-data/human_eval/afafd8eb-20e3-44ca-8dec-fe2c83fe42f7.py
LaudateCorpus1/code-align-evals-data/human_eval/below_threshold.py
LaudateCorpus1/code-align-evals-data/human_eval/c26981da-a5f7-4e1d-910e-b77c11f50308.py
LaudateCorpus1/code-align-evals-data/human_eval/car_race.py
LaudateCorpus1/code-align-evals-data/human_eval/correct_bracketing.py
LaudateCorpus1/code-align-evals-data/human_eval/correct_bracketing2.py
LaudateCorpus1/code-align-evals-data/human_eval/d3977f22-d6ee-428e-b9ee-df4be3612155.py
LaudateCorpus1/code-align-evals-data/human_eval/db78737a-3a71-4616-80c9-a64e9ff8f360.py
LaudateCorpus1/code-align-evals-data/human_eval/de860404-0aa2-4de5-af73-b3d2e9895e0f.py
LaudateCorpus1/code-align-evals-data/human_eval/debfd7c5-1924-4952-b044-9f37bb60b0ae.py
LaudateCorpus1/code-align-evals-data/human_eval/decode_cyclic.py
LaudateCorpus1/code-align-evals-data/human_eval/derivative.py
LaudateCorpus1/code-align-evals-data/human_eval/eb01e967-8556-4002-81cf-2072eb0d8398.py
LaudateCorpus1/code-align-evals-data/human_eval/ee7cad63-ad3f-4d92-be25-c83f7189acf1.py
LaudateCorpus1/code-align-evals-data/human_eval/ef224818-fd57-4f43-942b-766be1e4842c.py
LaudateCorpus1/code-align-evals-data/human_eval/fcc9091a-a49a-46db-98a6-1ec1fbfbedbd.py
LaudateCorpus1/code-align-evals-data/human_eval/fdd7c74c-ff1c-4fe9-b5f5-c42039ce9d5f.py
LaudateCorpus1/code-align-evals-data/human_eval/fib.py
LaudateCorpus1/code-align-evals-data/human_eval/fibfib.py
LaudateCorpus1/code-align-evals-data/human_eval/find_zero.py
LaudateCorpus1/code-align-evals-data/human_eval/floats_find_closest_elements.py
LaudateCorpus1/code-align-evals-data/human_eval/floats_has_close_elements.py
LaudateCorpus1/code-align-evals-data/human_eval/floats_mean_absolute_deviation.py
LaudateCorpus1/code-align-evals-data/human_eval/floats_truncate_number.py
LaudateCorpus1/code-align-evals-data/human_eval/get_positive.py
LaudateCorpus1/code-align-evals-data/human_eval/incr_list.py
LaudateCorpus1/code-align-evals-data/human_eval/int_list_below_zero.py
LaudateCorpus1/code-align-evals-data/human_eval/int_list_remove_duplicates.py
LaudateCorpus1/code-align-evals-data/human_eval/ints_factorize.py
LaudateCorpus1/code-align-evals-data/human_eval/is_prime.py
LaudateCorpus1/code-align-evals-data/human_eval/largest_prime_factor.py
LaudateCorpus1/code-align-evals-data/human_eval/max.py
LaudateCorpus1/code-align-evals-data/human_eval/pairs_sum_to_zero.py
LaudateCorpus1/code-align-evals-data/human_eval/parsing_parse_music.py
LaudateCorpus1/code-align-evals-data/human_eval/parsing_parse_nested_parens.py
LaudateCorpus1/code-align-evals-data/human_eval/parsing_separate_paren_groups.py
LaudateCorpus1/code-align-evals-data/human_eval/prime_fib.py
LaudateCorpus1/code-align-evals-data/human_eval/remove_vowels.py
LaudateCorpus1/code-align-evals-data/human_eval/sort_even.py
LaudateCorpus1/code-align-evals-data/human_eval/strings_sort_numbers.py
LaudateCorpus1/code-align-evals-data/human_eval/triples_sum_to_zero.py
LaudateCorpus1/code-align-evals-data/human_eval/unique.py
Lcoderfit/Introduction-to-algotithms/十二、剑指offer-Python/4.二维数组中的查找.py
Leesoar/leetcode/easy/14.Longest_Common_Prefix.py
LingruiZhu/pyldpc-master/pyldpc/evaluation.py
Lioncat2002/AoC/day5/day5_part1.py
Lioncat2002/CSE-programs/python_programs/programs12062021/substringpos.py
M2000h/Algorithms/Sorts/Bubble Sort/Bubble Sort.py
Manasranjanpati/Intern_training/Python-Programs/sum_of_integer_using_recursion.py
MayaScarlet/leetcode-python/13-roman-to-integer/13-roman-to-integer.py
Mpreyzner/python_experiments/14_collatz_conjecture.py
Naouak/illacceptanything/code/palindrome.py
Neetansh/HACKTOBERFEST2021_PATTERN/Integer to Roman.py
Nitin-Diwakar/100-days-of-code/day44/problem.py
Perry961002/Algorithm-LeetCode/Python/012-integer-to-roman.py
PeteCoward/teach-python/easy_crypto/lesson2/task3.py
PraghadeshManivannan/Python/loop/armstrong number from 1 to n.py
Pratham82/Python-Programming/Practice_problems/Edabit/Recurssion sum.py
ProjektOsmium/All-Algorithms/coin_change_problem/Python/coinchange.py
QinganZhao/LXXtCode/LeetCode/012. Integer to Roman.py
RafaelHuang87/Leet-Code-Practice/12.py
Rahul-k25/python-cp-cheatsheet/leet/array/intToRoman.py
RideGreg/LeetCode/Python/find-all-anagrams-in-a-string.py
Robert-Ma/Python_Exercises/Search_and_Sorting/Exercise01.py
RobertElias/PythonProjects/Arrays/main_16.py
Sax-Ted/Practicing-with-Leetcode/0012-Integer-to-Roman.py
Siddharth1698/Deep-Learning-/Neural Networks/SoftMax.py
SleepNoMore/CodingBat/List-2/has22.py
Sourav692/100-Days-of-Python-Code/Day 1/8. Concert_roman_to_int.py
Sourav692/100-Days-of-Python-Code/Day 1/9. convert_int_to_roman.py
SreeshaKS/ds_algorithms/DS_Algo_Python/foobar/the-grandest-staircase-of-them-all/solution.py
SympleTechnologies/moov_backend/api/helper/camel_to_snake.py
TestowanieAutomatyczneUG/laboratorium-7-cati97/src/sample/zad2.py
TestowanieAutomatyczneUG/laboratorium_13-cati97/src/sample/roman_class.py
TestowanieAutomatyczneUG/laboratorium_14-cati97/src/sample/roman_class.py
abhisek-technicise/Python-Training/abhisek/Find_Adverbs.py
abhisek-technicise/Python-Training/abhisek/Sum_of_digits.py
abhisek-technicise/Python-Training/abhisek/matches a word at end of string, with optional punctuation.py
abhisek-technicise/Python-Training/abhisek/power.py
adi01trip01/adi_workspace/sample_problems/problems_with_solution15.py
aisolab/codility_lessons/Maximum_slice_problem/concept.py
aiuzunov/training-projects/tasks/self-learning/Python/Classes/intToRoman.py
andrea-ortalda/CarND-Traffic-Sign-Classifier-Project/utils/softmax.py
andriidem308/python_practice/matfiz_tasks/Topic9_Functions/7.40e.py
anjiadhikari/allPythonPractice/factorial.py
ankitrajbiswal/SEM_5/PIP/Minor Assignment 10/A10Q1.py
arifamd00/Python/Src/EvenorOdd.py
aurora314156/leetcode/tmp1.py
bgoonz/INTERVIEW-PREP-COMPLETE/notes-n-resources/Data-Structures-N-Algo/_DS-n-Algos/Interview-Problems/LeetCode/IntegerToRoman.py
bhupendpatil/Practice/Deep Learning/Neural Networks/Intro to Neural Networks/softmax.py
binggu56/qmd/GHQT/test.py
bpbpublications/Python-Quick-Interview-Guide/Chapter 01/int_roman.py
bpbpublications/TEST-YOUR-SKILLS-IN-PYTHON-LANGUAGE/Chapter 06/ch6_2_6.py
bprice5318/ia241-github-1/lec5.py
caofanCPU/Python/project_euler/problem_14/sol2.py
chars32/edx_python/Weeks/Week6/string_practices5.py
code-lgtm/Problem-Solving/python/leet_code/pow.py
coolsgupta/machine_learning_nanodegree/Deep_Learning/Deep_neural_networks/softmax.py
ctc316/algorithm-python/Lintcode/Ladder_1_Algorithm/02. Binary Search & LogN Algorithm/63. Search in Rotated Sorted Array II.py
ctc316/algorithm-python/Lintcode/Ladder_all_G_OA/1631. Interesting Subarray.py
d-biehl/robotcode/tests/robotcode/language_server/robotframework/tools.py
darlcruz/python-challenge-solutions/Darlington/phase-3/python class/day 99 solution/qtn1.py
darlcruz/python-challenge-solutions/Darlington/phase-3/python class/day 99 solution/qtn2.py
dexterpuru/leetcode_python/HashMap/Integer to Roman/Solution.py
dt-wark/codewars_practice/valid_parentheses.py
dynamoh/Interview-solved/Important/Fibonacci_series.py
emehrawn/Python-Codes/Class-Example/CLASSES_IN_PYTHON_13.py
faridghafoori/DS_Projects/CA2/farid-ghafooriadl-2.py
fbobee/Alpenglow/sip/scripts/sipcode.py
fhansmann/coding-challenges/c__13.py
francium/microurl/random_micro.py
frenchu/python-first-steps/pimontecarlo/__init__.py
geemaple/algorithm/leetcode/12.integer-to-roman.py
ghanender-chauhan/LeetCode-Solutions/Python/Integer TO Roman.py
hgbarrow/PythonSnippets/nested_struct_comp.py
hxz1998/funny_leetcode/Python/No12.py
iFun/Project-Euler/3.Largest prime factor.py
iml1111/programmers_coding_study/src/level2/게임맵최단거리.py
jdalzatec/EulerProject/Problem_7/main.py
joez/letspy/fun/max-consecutive.py
kannan5/LeetCode/1470_Shuffle_array.py
kbaddepuri/DataStructuresInPython/DataStructuresandAlgorithmsInPython/com/kranthi/Algorithms/challenges/gcd.py
kbrose/project_euler/p50-59/p52.py
keitheck/algorithm-and-solution-library/valid_parentheses.py
kkcookies99/UAST/Dataset/Leetcode/test/12/413.py
kkcookies99/UAST/Dataset/Leetcode/test/12/586.py
kkcookies99/UAST/Dataset/Leetcode/test/13/893.py
kkcookies99/UAST/Dataset/Leetcode/train/12/290.py
kkcookies99/UAST/Dataset/Leetcode/train/12/583.py
kkcookies99/UAST/Dataset/Leetcode/train/12/606.py
kkcookies99/UAST/Dataset/Leetcode/train/12/620.py
kkcookies99/UAST/Dataset/Leetcode/valid/12/417.py
kmyk/Jikka/examples/dp_min_square_kubaru.py
kmyk/Jikka/examples/dp_min_square_morau.py
kolyasalubov/Lv-677.PythonCore/HW7/Serhii_Yadzhak/2.py
kuldeep27396/DataStructuresandAlgorithmsInPython/com/kranthi/Algorithms/challenges/gcd.py
leonel-123/python-fundamentals/exercises/project_euler_solutions/problem_3.py
linkeshkanna/ProblemSolving/EDUREKA/Course.1-2/Case.Study.2.Programs/passwordvalidation.py
livioribeiro/project-euler/python/p003.py
lmonsalve22/Learning-to-Code/regExp/scripts/exercise_10.py
lunabox/leetcode/python/problems/s12_Integer_to_Roman.py
manoldonev/algo-challenges/src/lis_length.py
maximkir/project-euler/problems/p003.py
milkrong/Basic-Python-DS-Algs/LeetCode/int_to_roman.py
mishrakeshav/Competitive-Programming/binarysearch.io/matrix_search_sequal.py
mkoryor/Python/coding patterns/two pointers/remove_instance_of_key.py
netor27/codefights-arcade-solutions/arcade/python/arcade-intro/08_Diving Deeper/037_arrayMaxConsecutiveSum.py
nikitaKunevich/coding-training/Arrays/two_number_sum.py
okara83/Becoming-A-Data-Scientist/Data Science and Machine Learning/Machine-Learning-In-Python-THOROUGH/EXAMPLES/EDABIT/EARLIER/40_count_the_lone_ones.py
pallavimr12/Python_Levelwise_Exercises/Solution/13.py
paulsok/python/leetcode/int_to_roman.py
rcbyron/python-projects/competitions/gcd_two_numbers.py
shoaibur/SWE/Interview-Preparation/Facebook/Explore/01-Arrays-and-Strings/03-Integer-to-Roman.py
shuowangphd/lcpy/00/12.py
siddhartthecoder/pytorch-deep-learning-Udacity/Softmaxalgo2.15.py
simdevex/01.Basics/__numberOfDivsorsGivenInteger.py
sobil-dalal/Database-Analytical-Programming/Slides/3_mapAndReduce.py
steveflys/data-structures-and-algorithms/shift_array/shift_array.py
styam/coading_practice/HackerRank/repeated_string.py
styam/coading_practice/lambda/sort_list.py
sugia/leetcode/integer-to-roman.py
sumeyaali/Code_Challenges/Intro/rotateImage.py
tu2gin/algorithms-templates/python/sprint0/b.py
tumuum/prog-book/code/flag-example.py
ueg1990/aids/aids/strings/is_palindrome.py
unobatbayar/CodingBat/CodingBat/Python/List-2/has22.py
valcal/python_practice/python-function-practice-master/find_33.py
vishwesh5/Project-Euler/problem5.py
webdevhub42/Lambda/WEEKS/CD_Sata-Structures/_RESOURCES/CODESIGNAL/_ALL_PY/common_character_count.py
williamlagos/chess/solving/stacks/brackets.py
xianlinfeng/project_euler_python3/010.py
yalyakoob/Algorithmic-Coding-Challenges/Sliding Window/Maximum Sum Subarray of Size k.py
yanzhenchao/algorithms-in-python/search.py
ydavidchen/pytorch_dl_challenge/lessons_and_tutorials/02_intro_to_neuralnet/2.15_SoftMax.py
youngtech515/PythonScripts/Functional Programming/Sum_of_numbers_Recursion.py
zerohk/codingbat_py/List_2/has22.py
zhangrong1722/interview/code/at_offer/array/coding_interview21.py
zshashz/py1729/python-lab-file/69_factorialrecursion.py
loubnabnl commented 1 year ago

Thanks for working on this! These results are from the-stack, right? @RaymondLi0 found 336 files for HumanEval with exact matching. So only 15 were missed, we should probably identify them and see what they look like.

RaymondLi0 commented 1 year ago

Nicely done @ChenghaoMou ! I'm trying to see which files were not found with exact-match

RaymondLi0 commented 1 year ago

Among this list of 351 files you gave, actually 254 are in the dataset after I filtered the exact matches. This means that:

RaymondLi0 commented 1 year ago

This example doesn't really look like a MBPP or humanEval problem (fbobee/Alpenglow/sip/scripts/sipcode.py):

import sys
import re

if __name__ == "__main__":
  sip_code = False
  for line in sys.stdin:
    #sys.stderr.write("DEBUG: got line: " + line)
    start_sip_code = re.match("\s*/\* SIP_CODE", line)
    if start_sip_code:
      #sys.stderr.write("DEBUG: start sip_code\n")
      sip_code = True
    end_sip_code = sip_code and re.match("\s*\*/", line)
    if end_sip_code:
      #sys.stderr.write("DEBUG: end sip_code\n")
      sip_code = False
    if not start_sip_code and not end_sip_code :
      sys.stdout.write(line)
ChenghaoMou commented 1 year ago

@RaymondLi0 I made some changes:

  1. Look for duplicates based on HumanEval prompt + solution + test
  2. Look for duplicates based on MBPP text + code + test_list
  3. Lower the threshold to 0.58 (this is the minimum maximum Jaccard similarity for all LaudateCorpus1/code-align-evals-data/human_eval files against the openai_humaneval dataset.
  4. Double check Jaccard similarity to skip false positives

And here is the result

repository_name_path,dataset,task_id,jaccard_similarity
0xUvaish/The-Complete-FAANG-Preparation/3]. Competitive Programming/08]. LeetCode/1]. Problems/Python/0012)_i_Integer_to_Roman.py,mbpp,958,0.58
19251A05J0/Hacktoberfest2021/Python/convert_integer_to-roman_numeral.py,mbpp,958,0.68
AkantaRai/alecto/python assignments/function/q6_solution.py,mbpp,554,0.60
Akash671/LeetCode/Medium/number_to_roman.py,mbpp,958,0.69
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/LIST/day 41 solution/qtn8.py,mbpp,755,0.65
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/phase 1/python 2 basis/Day_23_Challenge_Solution/Question 9 Solution.py,mbpp,885,0.65
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/LIST/Day_41_Challenge_Solution/Question 8 Solution.py,mbpp,755,0.61
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/phase 1/python 2 basis/Day_26_Challenge_Solution/Question 4 Solution.py,mbpp,366,0.80
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/LIST/day 41 solution/qtn7.py,mbpp,755,0.81
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase1/python Basic 2/day 25 solution/qtn6.py,mbpp,801,0.79
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase1/python Basic 2/day 23 solution/qtn9.py,mbpp,885,0.68
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/LIST/Day_39_Challenge_Solution/Question 6 Solution.py,mbpp,896,0.68
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase1/python Basic 2/day 24 solution/qtn1.py,mbpp,131,0.70
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/LIST/day 39 solution/qtn6.py,mbpp,896,0.67
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 2/LIST/Day45 Tasks/Task6.py,mbpp,805,0.70
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/STRINGS/day 34 solution/qtn3.py,mbpp,602,0.67
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/phase 1/python 2 basis/Day_24_Challenge_Solution/Question 1 Solution.py,mbpp,131,0.69
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 1/Python Basic 2/Day23 Tasks/Task8.py,mbpp,256,0.74
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/TUPLE/Day_54_Challenge_Solution/Question 4 Solution.py,mbpp,941,0.71
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 2/LIST/Day41 Tasks/Task7.py,mbpp,755,0.75
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/STRINGS/day 34 solution/qtn10.py,mbpp,526,0.63
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 2/STRINGS/Day34 Tasks/Task5.py,mbpp,537,0.74
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 1/Python Basic 2/Day25 Tasks/Task9.py,mbpp,340,0.74
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/TUPLE/day 52 solution/qtn10.py,mbpp,405,0.59
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 1/Python Basic 2/Day26 Tasks/Task4.py,mbpp,366,0.76
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 2/LIST/Day41 Tasks/Task8.py,mbpp,755,0.65
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/Data Structure/day 60 solution/qtn4.py,mbpp,736,0.76
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase1/python Basic 2/day 26 solution/qtn4.py,mbpp,366,0.78
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase1/python Basic 2/day 25 solution/qtn9.py,mbpp,340,0.77
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/TUPLE/Day_54_Challenge_Solution/Question 3 Solution.py,mbpp,370,0.67
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/phase 1/python 2 basis/Day_26_Challenge_Solution/Question 5 Solution.py,mbpp,775,0.66
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 2/STRINGS/Day34 Tasks/Task3.py,mbpp,602,0.62
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/phase 1/python 2 basis/Day_19_Challenge_Solution/Question 9 Solution.py,mbpp,501,0.71
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 1/Python Basic 2/Day18 Tasks/Task4.py,mbpp,96,0.74
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/LIST/day 45 solution/qtn6.py,mbpp,805,0.75
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/phase 1/python 2 basis/Day_23_Challenge_Solution/Question 8 Solution.py,mbpp,256,0.70
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase1/python Basic 2/day 19 solution/qtn9.py,mbpp,501,0.75
CodedLadiesInnovateTech/-python-challenge-solutions/Ekeopara_Praise/Phase 2/DICTIONARY/Day48 Tasks/Task4.py,mbpp,703,0.67
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/STRINGS/day 34 solution/qtn5.py,mbpp,537,0.81
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/LIST/Day_41_Challenge_Solution/Question 7 Solution.py,mbpp,755,0.65
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/phase 1/python 2 basis/Day_25_Challenge_Solution/Question 9 Solution.py,mbpp,340,0.79
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/TUPLE/Day_52_Challenge_Solution/Question 10 Solution.py,mbpp,405,0.62
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/STRINGS/Day_34_Challenge_Solution/Question 3 Solution.py,mbpp,602,0.70
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/TUPLE/day 54 solution/qtn3.py,mbpp,370,0.74
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase2/Data Structure/day 60 solution/qtn5.py,mbpp,786,0.76
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/STRINGS/Day_34_Challenge_Solution/Question 5 Solution.py,mbpp,537,0.81
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/DICTIONARY/Day_48_Challenge_Solution/Question 4 Solution.py,mbpp,703,0.70
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase1/python Basic 2/day 27 solution/qtn3.py,mbpp,525,0.62
CodedLadiesInnovateTech/-python-challenge-solutions/Aniyom Ebenezer/Phase 2/STRINGS/Day_34_Challenge_Solution/Question 9 Solution.py,mbpp,480,0.68
CodedLadiesInnovateTech/-python-challenge-solutions/Uche Clare/Phase 2/String/Day 34/Task 9.py,mbpp,480,0.59
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase1/python Basic 2/day 19 solution/qtn4.py,mbpp,96,0.71
CodedLadiesInnovateTech/-python-challenge-solutions/Darlington/phase1/python Basic 2/day 23 solution/qtn8.py,mbpp,256,0.78
DefiniteError/randomCode/OCRChallenges/67-romanNumerals.py,mbpp,958,0.75
Divya333/Number-Theory/Python/List/program to get a list, sorted in increasing order by the last element in each tuple from a given list of non-empty tuples.py,mbpp,896,0.62
EmilianStoyanov/150_exercises_with_solution/list/6.py,mbpp,896,0.60
Kalpavrikshika/python_modules/inr_to_roman.py,mbpp,958,0.71
LaudateCorpus1/code-align-evals-data/human_eval/pairs_sum_to_zero.py,openai_humaneval,HumanEval/43,0.77
LaudateCorpus1/code-align-evals-data/human_eval/int_list_filter_integers.py,openai_humaneval,HumanEval/22,0.92
LaudateCorpus1/code-align-evals-data/human_eval/car_race.py,openai_humaneval,HumanEval/41,0.95
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/below_threshold.py,openai_humaneval,HumanEval/52,0.83
LaudateCorpus1/code-align-evals-data/human_eval/get_positive.py,openai_humaneval,HumanEval/30,0.88
LaudateCorpus1/code-align-evals-data/human_eval/floats_has_close_elements.py,openai_humaneval,HumanEval/0,0.93
LaudateCorpus1/code-align-evals-data/human_eval/260296e8-2f6d-4817-8555-2ff45aa9cf19.py,openai_humaneval,HumanEval/92,0.92
LaudateCorpus1/code-align-evals-data/human_eval/9c3ce973-d3a9-4988-9bdb-82f2cc304910.py,openai_humaneval,HumanEval/65,0.94
LaudateCorpus1/code-align-evals-data/human_eval/floats_truncate_number.py,openai_humaneval,HumanEval/2,0.92
LaudateCorpus1/code-align-evals-data/human_eval/7808faf6-4010-4c77-aa7d-532e0f311a08.py,openai_humaneval,HumanEval/152,0.97
LaudateCorpus1/code-align-evals-data/human_eval/56a7cf21-6ba9-45c0-8a1e-5165396b2ff2.py,openai_humaneval,HumanEval/117,0.92
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/count_nums.py,openai_humaneval,HumanEval/108,0.84
LaudateCorpus1/code-align-evals-data/human_eval/incr_list.py,openai_humaneval,HumanEval/42,0.88
LaudateCorpus1/code-align-evals-data/human_eval/7cc4fcfb-4513-407d-91d4-a6c1728abf44.py,openai_humaneval,HumanEval/129,0.97
LaudateCorpus1/code-align-evals-data/human_eval/f7270515-e834-4643-bc67-a3e2c803b8f4.py,openai_humaneval,HumanEval/100,0.85
LaudateCorpus1/code-align-evals-data/human_eval/5abba23f-d2af-4d07-ae71-c80d75641b2c.py,openai_humaneval,HumanEval/110,0.96
LaudateCorpus1/code-align-evals-data/human_eval/2363a4de-667f-485f-b3b9-860633d992de.py,openai_humaneval,HumanEval/97,0.95
LaudateCorpus1/code-align-evals-data/bad-solutions/below_zero.py,openai_humaneval,HumanEval/3,0.97
LaudateCorpus1/code-align-evals-data/human_eval/72ec63ac-d718-4dd9-8500-30a659f3177d.py,openai_humaneval,HumanEval/138,0.85
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/circular_shift.py,openai_humaneval,HumanEval/65,0.92
LaudateCorpus1/code-align-evals-data/alignment/find_bug/move_one_ball.py,openai_humaneval,HumanEval/109,0.67
LaudateCorpus1/code-align-evals-data/human_eval/9b6acd41-dd19-43a5-8960-486d31abc07c.py,openai_humaneval,HumanEval/93,0.96
LaudateCorpus1/code-align-evals-data/human_eval/ef224818-fd57-4f43-942b-766be1e4842c.py,openai_humaneval,HumanEval/81,0.92
LaudateCorpus1/code-align-evals-data/human_eval/12961850-50b0-44d8-861e-b7a4003efd6f.py,openai_humaneval,HumanEval/156,0.88
LaudateCorpus1/code-align-evals-data/human_eval/fdd7c74c-ff1c-4fe9-b5f5-c42039ce9d5f.py,openai_humaneval,HumanEval/78,0.97
LaudateCorpus1/code-align-evals-data/alignment/find_bug/select_words.py,openai_humaneval,HumanEval/117,0.60
LaudateCorpus1/code-align-evals-data/bad-solutions/count_upper.py,openai_humaneval,HumanEval/98,0.96
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/compare_one.py,openai_humaneval,HumanEval/137,0.92
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/by_length.py,openai_humaneval,HumanEval/105,0.88
LaudateCorpus1/code-align-evals-data/human_eval/bfc2b4bd-4729-49bc-8157-1544b0a8e989.py,openai_humaneval,HumanEval/108,0.96
LaudateCorpus1/code-align-evals-data/human_eval/506b24dd-a3f8-487d-ac5f-7756e3fb18b3.py,openai_humaneval,HumanEval/140,0.95
LaudateCorpus1/code-align-evals-data/human_eval/75270640-7b14-44ad-b70c-98bf8ea73879.py,openai_humaneval,HumanEval/135,0.58
LaudateCorpus1/code-align-evals-data/human_eval/afd3c4ce-baf5-45d5-bd82-5750e173d040.py,openai_humaneval,HumanEval/128,0.95
LaudateCorpus1/code-align-evals-data/bad-solutions/concatenate.py,openai_humaneval,HumanEval/28,0.92
LaudateCorpus1/code-align-evals-data/human_eval/floats_find_closest_elements.py,openai_humaneval,HumanEval/20,0.94
LaudateCorpus1/code-align-evals-data/bad-solutions/check_if_last_char_is_a_letter.py,openai_humaneval,HumanEval/134,0.95
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/can_arange.py,openai_humaneval,HumanEval/135,0.58
LaudateCorpus1/code-align-evals-data/human_eval/int_list_below_zero.py,openai_humaneval,HumanEval/3,0.94
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/bf.py,openai_humaneval,HumanEval/148,0.94
LaudateCorpus1/code-align-evals-data/human_eval/2664c3d1-bd10-44c0-a339-fe3cf75bc375.py,openai_humaneval,HumanEval/82,0.94
LaudateCorpus1/code-align-evals-data/human_eval/largest_prime_factor.py,openai_humaneval,HumanEval/59,0.85
LaudateCorpus1/code-align-evals-data/human_eval/max.py,openai_humaneval,HumanEval/35,0.84
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/concatenate.py,openai_humaneval,HumanEval/28,0.92
LaudateCorpus1/code-align-evals-data/bad-solutions/closest_integer.py,openai_humaneval,HumanEval/99,0.91
LaudateCorpus1/code-align-evals-data/human_eval/median.py,openai_humaneval,HumanEval/47,0.71
LaudateCorpus1/code-align-evals-data/bad-solutions/any_int.py,openai_humaneval,HumanEval/92,0.89
LaudateCorpus1/code-align-evals-data/human_eval/d4c45449-db7a-41bf-af53-bd1e19f82f49.py,openai_humaneval,HumanEval/162,0.92
LaudateCorpus1/code-align-evals-data/bad-solutions/add.py,openai_humaneval,HumanEval/85,0.96
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/compare.py,openai_humaneval,HumanEval/152,0.92
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/count_distinct_characters.py,openai_humaneval,HumanEval/16,0.91
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/change_base.py,openai_humaneval,HumanEval/44,0.94
LaudateCorpus1/code-align-evals-data/bad-solutions/by_length.py,openai_humaneval,HumanEval/105,0.91
LaudateCorpus1/code-align-evals-data/alignment/find_bug/is_multiply_prime.py,openai_humaneval,HumanEval/75,0.60
LaudateCorpus1/code-align-evals-data/bad-solutions/anti_shuffle.py,openai_humaneval,HumanEval/86,0.97
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/by_length.py,openai_humaneval,HumanEval/105,0.91
LaudateCorpus1/code-align-evals-data/human_eval/078e0e67-885a-469c-9cbb-61b4e0ac4a48.py,openai_humaneval,HumanEval/83,0.93
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/count_distinct_characters.py,openai_humaneval,HumanEval/16,0.76
LaudateCorpus1/code-align-evals-data/human_eval/strings_make_palindrome.py,openai_humaneval,HumanEval/10,0.93
LaudateCorpus1/code-align-evals-data/human_eval/6925d65c-c49e-4853-80a2-f9e003b33641.py,openai_humaneval,HumanEval/116,0.95
LaudateCorpus1/code-align-evals-data/human_eval/4ba255a6-3757-4526-9565-0d5ff760222c.py,openai_humaneval,HumanEval/109,0.97
LaudateCorpus1/code-align-evals-data/human_eval/43fd874c-47db-42f0-961c-8cafb6ed2bd4.py,openai_humaneval,HumanEval/71,0.95
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/cycpattern_check.py,openai_humaneval,HumanEval/154,0.87
LaudateCorpus1/code-align-evals-data/human_eval/db78737a-3a71-4616-80c9-a64e9ff8f360.py,openai_humaneval,HumanEval/89,0.95
LaudateCorpus1/code-align-evals-data/human_eval/parsing_separate_paren_groups.py,openai_humaneval,HumanEval/1,0.94
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/anti_shuffle.py,openai_humaneval,HumanEval/86,0.94
LaudateCorpus1/code-align-evals-data/bad-solutions/check_dict_case.py,openai_humaneval,HumanEval/95,0.89
LaudateCorpus1/code-align-evals-data/human_eval/744e9661-e012-41fa-aefe-2d75d62c3a3c.py,openai_humaneval,HumanEval/159,0.96
LaudateCorpus1/code-align-evals-data/bad-solutions/change_base.py,openai_humaneval,HumanEval/44,0.94
LaudateCorpus1/code-align-evals-data/human_eval/b1ddef23-aeb7-4a67-b1b6-61c1488ddc94.py,openai_humaneval,HumanEval/105,0.89
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/add_elements.py,openai_humaneval,HumanEval/122,0.92
LaudateCorpus1/code-align-evals-data/human_eval/149267b6-8503-428b-98ad-0a99088eca2e.py,openai_humaneval,HumanEval/149,0.89
LaudateCorpus1/code-align-evals-data/human_eval/636bffed-7f29-4c7b-a5fd-7823b8e412f5.py,openai_humaneval,HumanEval/91,0.96
LaudateCorpus1/code-align-evals-data/human_eval/unique.py,openai_humaneval,HumanEval/34,0.84
LaudateCorpus1/code-align-evals-data/human_eval/ints_factorize.py,openai_humaneval,HumanEval/25,0.94
LaudateCorpus1/code-align-evals-data/alignment/find_bug/minPath.py,openai_humaneval,HumanEval/129,0.76
LaudateCorpus1/code-align-evals-data/bad-solutions/cycpattern_check.py,openai_humaneval,HumanEval/154,0.85
LaudateCorpus1/code-align-evals-data/human_eval/int_list_remove_duplicates.py,openai_humaneval,HumanEval/26,0.92
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/below_threshold.py,openai_humaneval,HumanEval/52,0.95
LaudateCorpus1/code-align-evals-data/human_eval/correct_bracketing.py,openai_humaneval,HumanEval/56,0.89
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/count_upper.py,openai_humaneval,HumanEval/98,0.91
LaudateCorpus1/code-align-evals-data/bad-solutions/bf.py,openai_humaneval,HumanEval/148,0.97
LaudateCorpus1/code-align-evals-data/bad-solutions/add_elements.py,openai_humaneval,HumanEval/122,0.92
LaudateCorpus1/code-align-evals-data/human_eval/07f9a601-d04c-495c-bf85-9216e0d06794.py,openai_humaneval,HumanEval/119,0.83
LaudateCorpus1/code-align-evals-data/human_eval/153e7012-0524-4204-9f7c-3f0276bcc139.py,openai_humaneval,HumanEval/73,0.94
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/add_elements.py,openai_humaneval,HumanEval/122,0.89
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/check_if_last_char_is_a_letter.py,openai_humaneval,HumanEval/134,0.97
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/any_int.py,openai_humaneval,HumanEval/92,0.91
LaudateCorpus1/code-align-evals-data/human_eval/a0cbfb2f-4200-4006-8dfd-ce4dfba4e973.py,openai_humaneval,HumanEval/101,0.95
LaudateCorpus1/code-align-evals-data/human_eval/debfd7c5-1924-4952-b044-9f37bb60b0ae.py,openai_humaneval,HumanEval/143,0.96
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/count_up_to.py,openai_humaneval,HumanEval/96,0.69
LaudateCorpus1/code-align-evals-data/human_eval/b8de0e69-a764-43b0-b87d-7d168ea5b066.py,openai_humaneval,HumanEval/132,0.73
LaudateCorpus1/code-align-evals-data/human_eval/efe4224e-cb63-403e-85b8-550a966cadbf.py,openai_humaneval,HumanEval/115,0.96
LaudateCorpus1/code-align-evals-data/human_eval/51b1be3f-d417-418d-9236-bf203e68cd76.py,openai_humaneval,HumanEval/134,0.94
LaudateCorpus1/code-align-evals-data/human_eval/ea23598d-70be-467c-83a9-7c139d9b049e.py,openai_humaneval,HumanEval/66,0.95
LaudateCorpus1/code-align-evals-data/human_eval/5a690b90-cb9c-461e-9db8-9064fdc8aaba.py,openai_humaneval,HumanEval/131,0.93
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/add.py,openai_humaneval,HumanEval/85,0.91
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/correct_bracketing.py,openai_humaneval,HumanEval/56,0.68
LaudateCorpus1/code-align-evals-data/human_eval/ints_largest_divisor.py,openai_humaneval,HumanEval/24,0.91
LaudateCorpus1/code-align-evals-data/human_eval/triples_sum_to_zero.py,openai_humaneval,HumanEval/40,0.84
LaudateCorpus1/code-align-evals-data/human_eval/a5ff4686-b6ce-4610-afed-9f4e8ad86201.py,openai_humaneval,HumanEval/124,0.92
LaudateCorpus1/code-align-evals-data/human_eval/monotonic.py,openai_humaneval,HumanEval/57,0.76
LaudateCorpus1/code-align-evals-data/alignment/find_bug/factorize.py,openai_humaneval,HumanEval/25,0.60
LaudateCorpus1/code-align-evals-data/bad-solutions/compare_one.py,openai_humaneval,HumanEval/137,0.82
LaudateCorpus1/code-align-evals-data/human_eval/6a183ce5-8d03-4fc0-a3c2-41e5b5be11b8.py,openai_humaneval,HumanEval/158,0.96
LaudateCorpus1/code-align-evals-data/human_eval/4c17158c-7b8e-44e4-a149-7c00846b43cb.py,openai_humaneval,HumanEval/161,0.96
LaudateCorpus1/code-align-evals-data/human_eval/c7093255-390a-4a19-b0cc-9ad7ac7cab77.py,openai_humaneval,HumanEval/103,0.96
LaudateCorpus1/code-align-evals-data/bad-solutions/choose_num.py,openai_humaneval,HumanEval/102,0.97
LaudateCorpus1/code-align-evals-data/bad-solutions/car_race_collision.py,openai_humaneval,HumanEval/41,0.98
LaudateCorpus1/code-align-evals-data/human_eval/b31e6706-884b-4de0-8ea7-e6bc97ea71b2.py,openai_humaneval,HumanEval/70,0.94
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/all_prefixes.py,openai_humaneval,HumanEval/14,0.77
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/car_race_collision.py,openai_humaneval,HumanEval/41,0.93
LaudateCorpus1/code-align-evals-data/bad-solutions/circular_shift.py,openai_humaneval,HumanEval/65,0.97
LaudateCorpus1/code-align-evals-data/human_eval/c26981da-a5f7-4e1d-910e-b77c11f50308.py,openai_humaneval,HumanEval/86,0.96
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/count_nums.py,openai_humaneval,HumanEval/108,0.86
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/below_zero.py,openai_humaneval,HumanEval/3,0.97
LaudateCorpus1/code-align-evals-data/human_eval/19282fa9-e1cc-4df3-baa6-e61da53b6fae.py,openai_humaneval,HumanEval/154,0.85
LaudateCorpus1/code-align-evals-data/human_eval/modp.py,openai_humaneval,HumanEval/49,0.82
LaudateCorpus1/code-align-evals-data/human_eval/23b09018-22b1-4e8e-b989-e73e429c61f6.py,openai_humaneval,HumanEval/123,0.96
LaudateCorpus1/code-align-evals-data/human_eval/floats_mean_absolute_deviation.py,openai_humaneval,HumanEval/4,0.92
LaudateCorpus1/code-align-evals-data/human_eval/fcc9091a-a49a-46db-98a6-1ec1fbfbedbd.py,openai_humaneval,HumanEval/125,0.88
LaudateCorpus1/code-align-evals-data/human_eval/strings_string_xor.py,openai_humaneval,HumanEval/11,0.94
LaudateCorpus1/code-align-evals-data/human_eval/aee120e7-d154-4fe4-8236-1b4f2c426906.py,openai_humaneval,HumanEval/147,0.94
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/choose_num.py,openai_humaneval,HumanEval/102,0.93
LaudateCorpus1/code-align-evals-data/human_eval/strings_flip_case.py,openai_humaneval,HumanEval/27,0.90
LaudateCorpus1/code-align-evals-data/human_eval/b47d0e85-1bcc-4140-9e4c-f85ca5a9f10a.py,openai_humaneval,HumanEval/74,0.88
LaudateCorpus1/code-align-evals-data/alignment/find_bug/get_odd_collatz.py,openai_humaneval,HumanEval/123,0.60
LaudateCorpus1/code-align-evals-data/human_eval/int_list_sum_product.py,openai_humaneval,HumanEval/8,0.93
LaudateCorpus1/code-align-evals-data/human_eval/find_zero.py,openai_humaneval,HumanEval/32,0.91
LaudateCorpus1/code-align-evals-data/human_eval/17042b66-3a2c-4560-8898-f1ad8a60e38a.py,openai_humaneval,HumanEval/99,0.95
LaudateCorpus1/code-align-evals-data/bad-solutions/below_threshold.py,openai_humaneval,HumanEval/52,0.92
LaudateCorpus1/code-align-evals-data/human_eval/add.py,openai_humaneval,HumanEval/53,0.88
LaudateCorpus1/code-align-evals-data/human_eval/d3977f22-d6ee-428e-b9ee-df4be3612155.py,openai_humaneval,HumanEval/67,0.83
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/common.py,openai_humaneval,HumanEval/58,0.85
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/check_dict_case.py,openai_humaneval,HumanEval/95,0.91
LaudateCorpus1/code-align-evals-data/human_eval/strings_count_distinct_characters.py,openai_humaneval,HumanEval/16,0.90
LaudateCorpus1/code-align-evals-data/human_eval/3c2ffb83-f154-49c0-bc89-4decd871a376.py,openai_humaneval,HumanEval/88,0.95
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/closest_integer.py,openai_humaneval,HumanEval/99,0.88
LaudateCorpus1/code-align-evals-data/human_eval/82fd6b25-b917-440e-8b61-7fbb1eda8a25.py,openai_humaneval,HumanEval/150,0.94
LaudateCorpus1/code-align-evals-data/human_eval/d800f411-34d4-4727-a688-074871dec462.py,openai_humaneval,HumanEval/141,0.97
LaudateCorpus1/code-align-evals-data/bad-solutions/compare.py,openai_humaneval,HumanEval/152,0.73
LaudateCorpus1/code-align-evals-data/human_eval/de860404-0aa2-4de5-af73-b3d2e9895e0f.py,openai_humaneval,HumanEval/118,0.96
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/compare_one.py,openai_humaneval,HumanEval/137,0.84
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/add.py,openai_humaneval,HumanEval/85,0.96
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/below_zero.py,openai_humaneval,HumanEval/3,0.85
LaudateCorpus1/code-align-evals-data/human_eval/strings_sort_numbers.py,openai_humaneval,HumanEval/19,0.94
LaudateCorpus1/code-align-evals-data/human_eval/6019e56a-5870-4f54-a3f9-b3ee73803311.py,openai_humaneval,HumanEval/85,0.93
LaudateCorpus1/code-align-evals-data/human_eval/7f20770f-78ed-4ee3-8177-d0d137ce87c9.py,openai_humaneval,HumanEval/87,0.96
LaudateCorpus1/code-align-evals-data/human_eval/0ae4724f-468c-42f7-83ee-5f70a99ee21d.py,openai_humaneval,HumanEval/157,0.95
LaudateCorpus1/code-align-evals-data/human_eval/fibfib.py,openai_humaneval,HumanEval/63,0.92
LaudateCorpus1/code-align-evals-data/human_eval/a7544525-91e6-4454-9b86-9d0df5f09874.py,openai_humaneval,HumanEval/77,0.80
LaudateCorpus1/code-align-evals-data/human_eval/7fd9b429-909e-43ef-bfe5-6ffda05bccea.py,openai_humaneval,HumanEval/111,0.95
LaudateCorpus1/code-align-evals-data/human_eval/cb920b10-c437-4da7-b7be-534600377af0.py,openai_humaneval,HumanEval/113,0.86
LaudateCorpus1/code-align-evals-data/human_eval/parsing_parse_music.py,openai_humaneval,HumanEval/17,0.94
LaudateCorpus1/code-align-evals-data/alignment/find_bug/parse_nested_parens.py,openai_humaneval,HumanEval/6,0.63
LaudateCorpus1/code-align-evals-data/human_eval/decode_shift.py,openai_humaneval,HumanEval/50,0.93
LaudateCorpus1/code-align-evals-data/human_eval/b0899b1a-07ef-48a9-b0e1-a2e6d44ceb6b.py,openai_humaneval,HumanEval/133,0.95
LaudateCorpus1/code-align-evals-data/human_eval/prime_fib.py,openai_humaneval,HumanEval/39,0.85
LaudateCorpus1/code-align-evals-data/human_eval/is_palindrome.py,openai_humaneval,HumanEval/48,0.88
LaudateCorpus1/code-align-evals-data/human_eval/eb01e967-8556-4002-81cf-2072eb0d8398.py,openai_humaneval,HumanEval/69,0.95
LaudateCorpus1/code-align-evals-data/human_eval/sort_even.py,openai_humaneval,HumanEval/37,0.90
LaudateCorpus1/code-align-evals-data/human_eval/1009d7d6-f904-4e69-b642-993f38ddb2da.py,openai_humaneval,HumanEval/95,0.92
LaudateCorpus1/code-align-evals-data/human_eval/same_chars.py,openai_humaneval,HumanEval/54,0.89
LaudateCorpus1/code-align-evals-data/bad-solutions/correct_bracketing.py,openai_humaneval,HumanEval/56,0.77
LaudateCorpus1/code-align-evals-data/human_eval/3d664914-2e5f-40d5-9237-4dc18f85827e.py,openai_humaneval,HumanEval/80,0.91
LaudateCorpus1/code-align-evals-data/human_eval/9163f3a4-d59f-4229-8a1c-064a33687069.py,openai_humaneval,HumanEval/136,0.94
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/cycpattern_check.py,openai_humaneval,HumanEval/154,0.82
LaudateCorpus1/code-align-evals-data/human_eval/7fd85f4b-be16-4616-93b9-2d4e2f7b8f4d.py,openai_humaneval,HumanEval/145,0.96
LaudateCorpus1/code-align-evals-data/human_eval/fib4.py,openai_humaneval,HumanEval/46,0.86
LaudateCorpus1/code-align-evals-data/human_eval/65343c25-d7c4-40c2-baeb-6d24506d4e0f.py,openai_humaneval,HumanEval/121,0.94
LaudateCorpus1/code-align-evals-data/human_eval/b4912a97-579f-4e95-8dfd-9e4278d23e97.py,openai_humaneval,HumanEval/84,0.95
LaudateCorpus1/code-align-evals-data/alignment/find_bug/car_race_collision.py,openai_humaneval,HumanEval/41,0.63
LaudateCorpus1/code-align-evals-data/human_eval/d98453f8-b46f-43c1-9322-7de672335b09.py,openai_humaneval,HumanEval/120,0.94
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/choose_num.py,openai_humaneval,HumanEval/102,0.97
LaudateCorpus1/code-align-evals-data/human_eval/2e113901-6625-42f4-a7fa-ded70dd09b5f.py,openai_humaneval,HumanEval/142,0.96
LaudateCorpus1/code-align-evals-data/human_eval/triangle_area.py,openai_humaneval,HumanEval/45,0.87
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/count_upper.py,openai_humaneval,HumanEval/98,0.96
LaudateCorpus1/code-align-evals-data/human_eval/strings_strlen.py,openai_humaneval,HumanEval/23,0.87
LaudateCorpus1/code-align-evals-data/bad-solutions/count_distinct_characters.py,openai_humaneval,HumanEval/16,0.91
LaudateCorpus1/code-align-evals-data/human_eval/strings_longest.py,openai_humaneval,HumanEval/12,0.93
LaudateCorpus1/code-align-evals-data/human_eval/int_list_intersperse.py,openai_humaneval,HumanEval/5,0.92
LaudateCorpus1/code-align-evals-data/human_eval/ints_gcd.py,openai_humaneval,HumanEval/13,0.90
LaudateCorpus1/code-align-evals-data/human_eval/strings_concatenate.py,openai_humaneval,HumanEval/28,0.90
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/check_if_last_char_is_a_letter.py,openai_humaneval,HumanEval/134,0.91
LaudateCorpus1/code-align-evals-data/human_eval/fizz_buzz.py,openai_humaneval,HumanEval/36,0.82
LaudateCorpus1/code-align-evals-data/human_eval/strings_all_prefixes.py,openai_humaneval,HumanEval/14,0.92
LaudateCorpus1/code-align-evals-data/human_eval/parsing_parse_nested_parens.py,openai_humaneval,HumanEval/6,0.94
LaudateCorpus1/code-align-evals-data/human_eval/3f0c3002-8ed7-4ac7-9d2c-ea7f60b50b00.py,openai_humaneval,HumanEval/151,0.90
LaudateCorpus1/code-align-evals-data/human_eval/strings_filter_by_substring.py,openai_humaneval,HumanEval/7,0.93
LaudateCorpus1/code-align-evals-data/human_eval/int_list_rolling_max.py,openai_humaneval,HumanEval/9,0.92
LaudateCorpus1/code-align-evals-data/alignment/find_bug/separate_paren_groups.py,openai_humaneval,HumanEval/1,0.59
LaudateCorpus1/code-align-evals-data/human_eval/607ba98e-fe4f-4bc5-bee9-5095e1e85bcb.py,openai_humaneval,HumanEval/90,0.93
LaudateCorpus1/code-align-evals-data/human_eval/c1a717ea-5d65-4d36-aebc-18fc17a0ec0c.py,openai_humaneval,HumanEval/79,0.95
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/change_base.py,openai_humaneval,HumanEval/44,0.87
LaudateCorpus1/code-align-evals-data/human_eval/ba02eaec-e8f8-4b89-8c05-fcf67c88d17f.py,openai_humaneval,HumanEval/160,0.96
LaudateCorpus1/code-align-evals-data/human_eval/ed0df9ba-a2f3-4bb1-a0ae-8c7fdcc25461.py,openai_humaneval,HumanEval/144,0.96
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/bf.py,openai_humaneval,HumanEval/148,0.98
LaudateCorpus1/code-align-evals-data/human_eval/0ae8a111-f96a-4d50-88bc-432c88fa563f.py,openai_humaneval,HumanEval/130,0.96
LaudateCorpus1/code-align-evals-data/human_eval/derivative.py,openai_humaneval,HumanEval/62,0.87
LaudateCorpus1/code-align-evals-data/human_eval/c3eb9ce8-6b01-4e90-b02e-5f0cb4d43694.py,openai_humaneval,HumanEval/137,0.95
LaudateCorpus1/code-align-evals-data/human_eval/d58f4a7e-6216-420d-8d57-9707eda53d14.py,openai_humaneval,HumanEval/126,0.94
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/car_race_collision.py,openai_humaneval,HumanEval/41,0.98
LaudateCorpus1/code-align-evals-data/human_eval/strings_filter_by_prefix.py,openai_humaneval,HumanEval/29,0.92
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/any_int.py,openai_humaneval,HumanEval/92,0.90
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/circular_shift.py,openai_humaneval,HumanEval/65,0.97
LaudateCorpus1/code-align-evals-data/human_eval/4fbd1e17-82eb-40d4-83fb-478e2d010cc9.py,openai_humaneval,HumanEval/76,0.94
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/all_prefixes.py,openai_humaneval,HumanEval/14,0.94
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/compare.py,openai_humaneval,HumanEval/152,0.77
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/closest_integer.py,openai_humaneval,HumanEval/99,0.98
LaudateCorpus1/code-align-evals-data/human_eval/is_prime.py,openai_humaneval,HumanEval/31,0.90
LaudateCorpus1/code-align-evals-data/human_eval/d72c2084-8060-432e-8754-70d421c28696.py,openai_humaneval,HumanEval/98,0.93
LaudateCorpus1/code-align-evals-data/human_eval/decode_cyclic.py,openai_humaneval,HumanEval/38,0.94
LaudateCorpus1/code-align-evals-data/alignment/find_bug/fruit_distribution.py,openai_humaneval,HumanEval/67,0.60
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/correct_bracketing.py,openai_humaneval,HumanEval/56,0.94
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/anti_shuffle.py,openai_humaneval,HumanEval/86,0.97
LaudateCorpus1/code-align-evals-data/alignment/find_bug/get_closest_vowel.py,openai_humaneval,HumanEval/118,0.59
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/bad_solutions/check_dict_case.py,openai_humaneval,HumanEval/95,0.87
LaudateCorpus1/code-align-evals-data/alignment/bad_contexts/good_solutions/common.py,openai_humaneval,HumanEval/58,0.95
LaudateCorpus1/code-align-evals-data/human_eval/sort_third.py,openai_humaneval,HumanEval/33,0.87
LaudateCorpus1/code-align-evals-data/human_eval/6c4a3470-1325-4552-8709-8fb0b727d387.py,openai_humaneval,HumanEval/107,0.96
LaudateCorpus1/code-align-evals-data/human_eval/ints_string_sequence.py,openai_humaneval,HumanEval/15,0.91
LaudateCorpus1/code-align-evals-data/human_eval/fib.py,openai_humaneval,HumanEval/55,0.72
LaudateCorpus1/code-align-evals-data/human_eval/602422cc-9954-4cad-9857-81e67967b878.py,openai_humaneval,HumanEval/148,0.96
LaudateCorpus1/code-align-evals-data/human_eval/202d0538-79a6-48a4-8743-439eccb46595.py,openai_humaneval,HumanEval/106,0.93
LaudateCorpus1/code-align-evals-data/bad-solutions/all_prefixes.py,openai_humaneval,HumanEval/14,0.92
LaudateCorpus1/code-align-evals-data/human_eval/3f98cad4-1bce-4401-9a2f-d01b5d33f373.py,openai_humaneval,HumanEval/96,0.67
LaudateCorpus1/code-align-evals-data/human_eval/bdff3490-5c1f-478e-b185-7a8e071b10d0.py,openai_humaneval,HumanEval/64,0.96
LaudateCorpus1/code-align-evals-data/human_eval/748ed056-f48b-4ba4-8c55-9af7fc01a9bf.py,openai_humaneval,HumanEval/146,0.95
LaudateCorpus1/code-align-evals-data/human_eval/ee7cad63-ad3f-4d92-be25-c83f7189acf1.py,openai_humaneval,HumanEval/94,0.91
LaudateCorpus1/code-align-evals-data/alignment/find_bug/find_closest_elements.py,openai_humaneval,HumanEval/20,0.60
LaudateCorpus1/code-align-evals-data/human_eval/ccefb901-689a-4ff7-a22f-b9436583920a.py,openai_humaneval,HumanEval/75,0.85
LaudateCorpus1/code-align-evals-data/human_eval/correct_bracketing2.py,openai_humaneval,HumanEval/56,0.89
LaudateCorpus1/code-align-evals-data/human_eval/5918fa1d-8ddd-4a93-8a62-6a434c0078af.py,openai_humaneval,HumanEval/163,0.94
LaudateCorpus1/code-align-evals-data/human_eval/change_base.py,openai_humaneval,HumanEval/44,0.92
LaudateCorpus1/code-align-evals-data/human_eval/strings_how_many_times.py,openai_humaneval,HumanEval/18,0.92
LaudateCorpus1/code-align-evals-data/human_eval/below_threshold.py,openai_humaneval,HumanEval/52,0.90
LaudateCorpus1/code-align-evals-data/human_eval/afafd8eb-20e3-44ca-8dec-fe2c83fe42f7.py,openai_humaneval,HumanEval/127,0.96
LaudateCorpus1/code-align-evals-data/human_eval/8e2ae8ed-1132-497c-9a3a-c4302bcbb8f5.py,openai_humaneval,HumanEval/153,0.97
LaudateCorpus1/code-align-evals-data/human_eval/eddfac58-792d-4560-b40c-505b663bad38.py,openai_humaneval,HumanEval/102,0.94
LaudateCorpus1/code-align-evals-data/human_eval/01b8dccc-a2a5-45f7-9e48-5691d0ee5257.py,openai_humaneval,HumanEval/68,0.96
LaudateCorpus1/code-align-evals-data/human_eval/common.py,openai_humaneval,HumanEval/58,0.90
LaudateCorpus1/code-align-evals-data/human_eval/floats_rescale_to_unit.py,openai_humaneval,HumanEval/21,0.94
LaudateCorpus1/code-align-evals-data/bad-solutions/common.py,openai_humaneval,HumanEval/58,0.95
LaudateCorpus1/code-align-evals-data/human_eval/e082c08a-e005-41fd-a052-1a6ea9bce8f4.py,openai_humaneval,HumanEval/122,0.90
LaudateCorpus1/code-align-evals-data/human_eval/06ee3e8f-68b4-4e17-8751-f1bd7d40f4fd.py,openai_humaneval,HumanEval/155,0.94
LaudateCorpus1/code-align-evals-data/human_eval/79a3ccd8-cab0-4c88-9a5f-b1f9d4c591de.py,openai_humaneval,HumanEval/104,0.90
LaudateCorpus1/code-align-evals-data/human_eval/37c941d1-78a2-46a4-89ce-a02d0d4c4c3e.py,openai_humaneval,HumanEval/114,0.95
LaudateCorpus1/code-align-evals-data/human_eval/2869a1e7-1d1a-4664-8077-421abbe852de.py,openai_humaneval,HumanEval/112,0.90
LaudateCorpus1/code-align-evals-data/human_eval/14240d5e-33ed-4be8-af2a-2d036ff5882b.py,openai_humaneval,HumanEval/139,0.87
LaudateCorpus1/code-align-evals-data/human_eval/3bec0102-1dd2-44a2-8506-420b7bdef9d1.py,openai_humaneval,HumanEval/72,0.94
LaudateCorpus1/code-align-evals-data/human_eval/sum_to_n.py,openai_humaneval,HumanEval/60,0.78
Manasranjanpati/Intern_training/Python-Programs/sum_of_integer_using_recursion.py,mbpp,566,0.60
Nitin-Diwakar/100-days-of-code/day44/problem.py,mbpp,822,0.59
Nitin-Diwakar/100-days-of-code/day50/problem.py,mbpp,129,0.74
Realocity/Python-Practicals/Practical 3/Q13.py,mbpp,703,0.60
Robert-Ma/Python_Exercises/Data_Structures/Exercise25.py,mbpp,786,0.68
RobertElias/PythonProjects/Arrays/main_15.py,mbpp,22,0.71
RobertElias/PythonProjects/Arrays/main_16.py,mbpp,74,0.82
Sourav692/100-Days-of-Python-Code/Day 1/9. convert_int_to_roman.py,mbpp,958,0.80
Sourav692/100-Days-of-Python-Code/Day 1/8. Concert_roman_to_int.py,mbpp,961,0.81
TestowanieAutomatyczneUG/laboratorium-7-cati97/src/sample/zad2.py,mbpp,958,0.75
TestowanieAutomatyczneUG/laboratorium_13-cati97/src/sample/roman_class.py,mbpp,958,0.75
TestowanieAutomatyczneUG/laboratorium_14-cati97/src/sample/roman_class.py,mbpp,958,0.75
abhisek-technicise/Python-Training/abhisek/power.py,mbpp,93,0.64
abhisek-technicise/Python-Training/abhisek/Sum_of_digits.py,mbpp,566,0.61
aiuzunov/training-projects/tasks/self-learning/Python/Classes/intToRoman.py,mbpp,958,0.67
darlcruz/python-challenge-solutions/Darlington/phase-3/python class/day 99 solution/qtn2.py,mbpp,961,0.80
darlcruz/python-challenge-solutions/Darlington/phase-3/python class/day 99 solution/qtn1.py,mbpp,958,0.79
emehrawn/Python-Codes/Class-Example/CLASSES_IN_PYTHON_13.py,mbpp,958,0.68
ghanender-chauhan/LeetCode-Solutions/Python/Integer TO Roman.py,mbpp,958,0.68
kkcookies99/UAST/Dataset/Leetcode/train/12/606.py,mbpp,958,0.58
kkcookies99/UAST/Dataset/Leetcode/train/12/583.py,mbpp,958,0.59
simdevex/01.Basics/__computeSumOfThreeLowestPositiveNumber.py,mbpp,340,0.62
simdevex/01.Basics/__numberOfDivsorsGivenInteger.py,mbpp,96,0.64
sugia/leetcode/integer-to-roman.py,mbpp,958,0.63

distribution over repos

repository_name
0xUvaish/The-Complete-FAANG-Preparation                  1
19251A05J0/Hacktoberfest2021                             1
AkantaRai/alecto                                         1
Akash671/LeetCode                                        1
CodedLadiesInnovateTech/-python-challenge-solutions     48
DefiniteError/randomCode                                 1
Divya333/Number-Theory                                   1
EmilianStoyanov/150_exercises_with_solution              1
Kalpavrikshika/python_modules                            1
LaudateCorpus1/code-align-evals-data                   250
Manasranjanpati/Intern_training                          1
Nitin-Diwakar/100-days-of-code                           2
Realocity/Python-Practicals                              1
Robert-Ma/Python_Exercises                               1
RobertElias/PythonProjects                               2
Sourav692/100-Days-of-Python-Code                        2
TestowanieAutomatyczneUG/laboratorium-7-cati97           1
TestowanieAutomatyczneUG/laboratorium_13-cati97          1
TestowanieAutomatyczneUG/laboratorium_14-cati97          1
abhisek-technicise/Python-Training                       2
aiuzunov/training-projects                               1
darlcruz/python-challenge-solutions                      2
emehrawn/Python-Codes                                    1
ghanender-chauhan/LeetCode-Solutions                     1
kkcookies99/UAST                                         2
simdevex/01.Basics                                       2
sugia/leetcode                                           1

distribution over datasets

duplicate_dataset
mbpp                 80
openai_humaneval    250
reshinthadithyan commented 1 year ago

Are there plans to look at other potential benchmarks like Code Contests Dataset from the Alphacode by Deepmind as well?

ChenghaoMou commented 1 year ago

@reshinthadithyan not that I am aware of. As far as I can see, CodeContests is released via GCP and has a riegeli suffix we don't include. Though, it is still possible that similar coding challenges from those source websites are present in the stack (just like how MBPP coincides with Leetcode). We will definitely double-check on that if needed.

ChenghaoMou commented 1 year ago

Close this as #14 and #16 are merged.