search

ErikBjare / gptme

Your agent in your terminal, equipped with local tools: writes code, uses the terminal, browses the web, vision.
https://gptme.org/docs/
MIT License
808 stars 62 forks source link

fix: made eval harness more reliable, using Manager #119

Closed ErikBjare closed 2 weeks ago

ellipsis-dev[bot] commented 2 weeks ago

Your free trial has expired. To keep using Ellipsis, sign up at https://app.ellipsis.dev for $20/seat/month or reach us at help@ellipsis.dev

codecov-commenter commented 2 weeks ago

Codecov Report

Attention: Patch coverage is 55.76923% with 23 lines in your changes missing coverage. Please review.

Project coverage is 81.74%. Comparing base (0208e76) to head (bf079eb). Report is 1 commits behind head on master.

:white_check_mark: All tests successful. No failed tests found.

Files with missing lines Patch % Lines
gptme/eval/main.py 57.14% 21 Missing :warning:
gptme/eval/suites/init_projects.py 33.33% 2 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #119 +/- ## ========================================== + Coverage 79.77% 81.74% +1.96% ========================================== Files 49 49 Lines 2903 2920 +17 ========================================== + Hits 2316 2387 +71 + Misses 587 533 -54 ``` | [Flag](https://app.codecov.io/gh/ErikBjare/gptme/pull/119/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Erik+Bj%C3%A4reholt) | Coverage Δ | | |---|---|---| | [anthropic](https://app.codecov.io/gh/ErikBjare/gptme/pull/119/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Erik+Bj%C3%A4reholt) | `78.28% <55.76%> (-0.29%)` | :arrow_down: | | [openai](https://app.codecov.io/gh/ErikBjare/gptme/pull/119/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Erik+Bj%C3%A4reholt) | `79.93% <50.00%> (+1.97%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Erik+Bj%C3%A4reholt#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.