server test coverage - Githubissues

matt-bernstein commented 2 months ago

Closes out: https://humansignal.atlassian.net/browse/DIA-1001 https://humansignal.atlassian.net/browse/DIA-1143 https://humansignal.atlassian.net/browse/DIA-1171

server tests with real server running and real calls to openai, behind pytest flags --use-server and --use-openai so they don't get autorun in CI
use a new CSVHandler with tempfiles to get results back to the caller in pytest so you can assert on expected output
use openai_responses lib for mocking openai
fix server settings
update readme with settings and pytest instructions
remove python 3.8 from supported versions due to use of asyncio.TaskGroup in server
use VCR.py to mock openai in local (non-server) pytests to replace custom mock functions
add test coverage for all skills

robot-ci-heartex commented 2 months ago

Commit https://github.com/HumanSignal/infra/commit/f78391a8b5f62af8df266b1ec8f362cad275ef43 is created.

robot-ci-heartex commented 2 months ago

Commit https://github.com/HumanSignal/infra/commit/cf038020e77a9a246478a603e6ddef7f893b2449 is created.

robot-ci-heartex commented 2 months ago

Commit https://github.com/HumanSignal/infra/commit/a337ee2072434148fc5253b6deef56883c1f572f is created.

robot-ci-heartex commented 2 months ago

Commit https://github.com/HumanSignal/infra/commit/0e6787d6f778e1d67173f6579b374c44e3c86eff is created.

robot-ci-heartex commented 2 months ago

Commit https://github.com/HumanSignal/infra/commit/2bbb589b39a3b6afc1219e1f8511905b4a0f3a33 is created.

robot-ci-heartex commented 2 months ago

Commit https://github.com/HumanSignal/infra/commit/8e73087d0066b180a14291a86726050fca20b8b8 is created.

codecov-commenter commented 2 months ago

Codecov Report

Attention: Patch coverage is 51.51515% with 16 lines in your changes missing coverage. Please review.

Project coverage is 55.87%. Comparing base (c199a33) to head (237590b). Report is 1 commits behind head on master.

Files	Patch %	Lines
server/handlers/result_handlers.py	38.09%	13 Missing :warning:
server/app.py	72.72%	3 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #141 +/- ## =========================================== + Coverage 45.71% 55.87% +10.16% =========================================== Files 40 40 Lines 1737 1754 +17 =========================================== + Hits 794 980 +186 + Misses 943 774 -169 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

robot-ci-heartex commented 2 months ago

Commit https://github.com/HumanSignal/infra/commit/65efb77c7d60506fa5fe366078a6a811f8fee3f3 is created.

robot-ci-heartex commented 2 months ago

Commit https://github.com/HumanSignal/infra/commit/3f4a6c6c69170c15e74f68ad117ff9093417cbd5 is created.

robot-ci-heartex commented 2 months ago

Commit https://github.com/HumanSignal/infra/commit/bc5083a40716cfcd3eafdcefe22c6fdd53d22bae is created.

robot-ci-heartex commented 2 months ago

Commit https://github.com/HumanSignal/infra/commit/df1c6b13f9ad95590e2c08eb93b7425abdba6861 is created.

robot-ci-heartex commented 2 months ago

Commit https://github.com/HumanSignal/infra/commit/6efd7519cf2ba4b19fa11d740c3e9be6ec30902b is created.

robot-ci-heartex commented 2 months ago