Question about metric implementation

[marcosmacedo]

Hi,

Thank you for sharing your CodeBLEU package. Could you please explain why is there an or 1 expression in the condition below in your implementation?

+ theta * (dataflow_match_score or 1)

https://github.com/k4black/codebleu/blob/4b8d576af9787bbc126db93df9d1e071898eaf85/codebleu/codebleu.py#L87

This is different from the XLCoST implementation.

https://github.com/reddy-lab-code-research/XLCoST/blob/ad46a7df51ea9e88f37a2f7e6edc5cbe4d13b2f2/code/translation/evaluator/CodeBLEU/calc_code_bleu.py#L76

During my testing I get a CodeBLEU score of 0.25 even if all the weights of the metric are zero. Is that the intended behavior?

Thank you

[k4black]

@marcosmacedo Hey, thank you for spotting this; TBH i do not remember =) If im not mistaken it relay on examples from the original articles, but let me check

On it