michaelfeil / infinity

Infinity is a high-throughput, low-latency REST API for serving text-embeddings, reranking models and clip
https://michaelfeil.github.io/infinity/
MIT License
1.31k stars 96 forks source link

Add `EngineArray` Multi-Model [1/3] #200

Closed michaelfeil closed 5 months ago

michaelfeil commented 5 months ago

Closes #13

119

As a follow-up, need to make sure to launch multiple models via the CLI. This requires some refactoring of the cli. This is not part of this PR.

codecov-commenter commented 5 months ago

Codecov Report

Attention: Patch coverage is 88.52459% with 7 lines in your changes are missing coverage. Please review.

Project coverage is 73.60%. Comparing base (59a13c0) to head (6abebe1). Report is 4 commits behind head on main.

Files Patch % Lines
libs/infinity_emb/infinity_emb/engine.py 87.87% 4 Missing :warning:
libs/infinity_emb/infinity_emb/infinity_server.py 86.36% 3 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #200 +/- ## ========================================== + Coverage 73.49% 73.60% +0.11% ========================================== Files 32 32 Lines 1811 1849 +38 ========================================== + Hits 1331 1361 +30 - Misses 480 488 +8 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.