AutoResearch / autodoc

MIT License
0 stars 1 forks source link

refactor: Do one prediction per input sequence, easier experimentation #27

Closed carlosgjs closed 7 months ago

carlosgjs commented 7 months ago

The prediction/evaluation interfaces used List[List[str]] as the prediction type to account for: a) batch inference, i.e. generate docs for multiple code snippets in one pass and b) generating multiple predictions per input (sampling). But we've now decided to do deterministic prediction (temp=0) so there's no need for the second level.

This PR changes to the prediction return type to List[str], which also makes the code simpler.

Two additional changes:

codecov-commenter commented 7 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (10294bc) 97.00% compared to head (6f2c5da) 97.17%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #27 +/- ## ========================================== + Coverage 97.00% 97.17% +0.16% ========================================== Files 3 3 Lines 167 177 +10 ========================================== + Hits 162 172 +10 Misses 5 5 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.