How do I specify different Settings in the code generation script? baseline, retrieval, retrieval w/ ref

amazon-science / cceval

CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion (NeurIPS 2023)

https://crosscodeeval.github.io/

Apache License 2.0

113 stars 17 forks source link

How do I specify different Settings in the code generation script? baseline, retrieval, retrieval w/ ref #21

Open kartikzheng opened 4 months ago

kartikzheng commented 4 months ago

hi，How do I specify different Settings in the code generation script? baseline, retrieval, retrieval w/ ref. I see that the data set only contains three different retrievers: bm25, UniXCoder and OpenAI Ada. thanks.

zijwang commented 2 months ago

use_crossfile_context controls whether it is a baseline or with retrieval. retrievaland retrieval w/ ref are noted in the filepath of the prompt file: former is rg1 and later is orcale.