k4black / codebleu

Pip compatible CodeBLEU metric implementation available for linux/macos/win
https://pypi.org/project/codebleu/
MIT License
66 stars 12 forks source link

fix: Clean up code #1

Closed maximus12793 closed 1 year ago

maximus12793 commented 1 year ago

Note: The original paper proposes a default weighting of 0.1,0.1,0.4,0.4,. Happy to make a PR for this or leave as-is.

k4black commented 1 year ago

Great! Thanks for the contribution! :3 Could you please check and fix the tests that failed now?

Regarding default weights - I just copied it from the original repo, but your suggestion makes sense! Happy if you'll make PR (but please note the default values in README with paper ref)

maximus12793 commented 1 year ago

Updated tests + added logging when applicable. So actually this scoring logic does seem OK due to the weighting scheme. (for standalone tests you could return none then validate but think its fine as-is).

Couldn't find the referenced test_code_x_glue_readme_examples but updated these based on my machines fp-precision.

codecov-commenter commented 1 year ago

Welcome to Codecov :tada:

Once merged to your default branch, Codecov will compare your coverage reports and display the results in this comment.

Thanks for integrating Codecov - We've got you covered :open_umbrella:

k4black commented 1 year ago

@maximus12793 Here it is - an example from readme https://github.com/microsoft/CodeXGLUE/blob/main/Code-Code/code-to-code-trans/example.png

maximus12793 commented 1 year ago

Hmm, yea unfortunately not sure how to repro those numbers for the second test or if they used different weights, language (c_sharp?) or what. Updated to pass linter, should be good for now!