Why do I get a warning when I use the score function with lang="en"

[higgsbosonprose]

I get the following warning when using the code given below

CODE

def compute_code_bert_nl(candidates: list[str], references: list[list[str]]):
    return score(candidates, references, lang="en", rescale_with_baseline=True)[
        -1
    ].tolist()

WARNING

Warning: Baseline not Found for microsoft/codebert-base-mlm on en at [c:\Python39\lib\site-packages\code_bert_score\rescale_baseline/en/microsoft/codebert-base-mlm.tsv](file:///C:/Python39/lib/site-packages/code_bert_score/rescale_baseline/en/microsoft/codebert-base-mlm.tsv)

Can you tell me why this happens and what can I do to fix this, I have been following your notebook example and you use the same stuff. Also thanks I enjoyed going through your paper and found it really cool that you have such a neat setup on Github! Really appreciate the effort and thanks for looking at my question!

[urialon]

Hi @higgsbosonprose , Thank you for your interest in our work and for your kind words!

I will fix that and let you know. In the meantime, you can run it without rescale_with_baseline=True, and manually rescale the scores using: (score - 0.7)/(1 - 0.7).

This is what causing the error, not the "en" language. But why are you using "en" ? The language should be the programming language, I.e. "java" / "python" etc.

Best, Uri

[higgsbosonprose]

Hey @urialon, Thanks for your prompt reply. I am working on a project where we are using CodeBERT score as a baseline for evaluating NL explanations of code. I want to get a score of generated explanations with reference to the Ground Truth explanation (also in NL) that is why I use "en" :)

Best, Saksham

[urialon]

Hi @higgsbosonprose , Thanks again for your feedback.

I set a default baseline to 0.5. This is updated in this repo and on pypi, so you can pip install code_bert_score --upgrade

Let us know if you have any more questions or feedback! Uri