k4black / codebleu

Pip compatible CodeBLEU metric implementation available for linux/macos/win
https://pypi.org/project/codebleu/
MIT License
66 stars 12 forks source link

TypeError: an integer is required #62

Open PiXianDouBanJiang opened 2 months ago

PiXianDouBanJiang commented 2 months ago

I failed on the given example with a typeerror.

TypeError Traceback (most recent call last) Cell In[1], line 6 3 prediction = "def add ( a , b ) :\n return a + b" 4 reference = "def sum ( first , second ) :\n return second + first" ----> 6 result = calc_codebleu([reference], [prediction], lang="python", weights=(0.25, 0.25, 0.25, 0.25), tokenizer=None) 7 print(result)

File d:\cc\LLM_DynaEval_Coding\mbpp+\codebleu\codebleu.py:41, in calc_codebleu(references, predictions, lang, weights, tokenizer, keywords_dir) 38 assert keywords_dir.exists(), f"keywords_dir {keywords_dir} does not exist" 40 # get the tree-sitter language for a given language ---> 41 tree_sitter_language = get_tree_sitter_language(lang) 43 # preprocess inputs 44 references = [[x.strip() for x in ref] if isinstance(ref, list) else [ref.strip()] for ref in references]

File d:\cc\LLM_DynaEval_Coding\mbpp+\codebleu\utils.py:163, in get_tree_sitter_language(lang) 160 elif lang == "python": 161 import tree_sitter_python --> 163 return Language(tree_sitter_python.language()) 164 elif lang == "go": 165 import tree_sitter_go

TypeError: an integer is required

chao-peng commented 2 months ago
pip install tree-sitter-python==0.21

will solve the problem.

ranimkhojah commented 1 month ago

pip install tree-sitter-python==0.21 did not work for me. @PiXianDouBanJiang did you find a way to make it work?

aur3l14no commented 1 month ago

I got it working with tree-sitter==0.23.1 and codebleu==0.7.1. The latter is not built and uploaded to PyPI yet, so you'd have to pip install git+https://github.com/k4black/codebleu#egg=codebleu.

raptech-jp commented 3 weeks ago

I’m also encountering the TypeError: an integer is required issue with tree-sitter==0.23.1 and codebleu==0.7.1. I tried pip install tree-sitter-python==0.21 as suggested by @chao-peng and installing CodeBLEU directly from GitHub as suggested by @aur3l14no, but neither resolved the issue. Has anyone found a reliable solution?

raptech-jp commented 2 weeks ago

I have an update. I wasn’t able to resolve the issue on my x86 Ubuntu environment, but I managed to get it working on an M3 MacBook with the following setup:

% pip freeze
codebleu==0.7.0
tree-sitter==0.23.2
tree-sitter-python==0.23.2
Weigang-Wu commented 6 days ago
pip install tree-sitter-python==0.21

will solve the problem.

Thanks,this method has solved my question.

nielstron commented 4 days ago

Since this is resolved by installing tree-sitter-0.23.1, the commit in https://github.com/k4black/codebleu/commit/bedf9e8aff668a8559551e7ba590937dfd4647f2 should resolve the issue. It is just not yet released.