Closed carlosgjs closed 7 months ago
All modified and coverable lines are covered by tests :white_check_mark:
Comparison is base (
10294bc
) 97.00% compared to head (6f2c5da
) 97.17%.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
The prediction/evaluation interfaces used
List[List[str]]
as the prediction type to account for: a) batch inference, i.e. generate docs for multiple code snippets in one pass and b) generating multiple predictions per input (sampling). But we've now decided to do deterministic prediction (temp=0) so there's no need for the second level.This PR changes to the prediction return type to
List[str]
, which also makes the code simpler.Two additional changes:
max_length
parameter in favor ofmax_new_tokens
eval
function intoeval_promp
for easier prompt experimentation