jina-ai / jina

☁️ Build multimodal AI applications with cloud-native stack
https://docs.jina.ai
Apache License 2.0
20.56k stars 2.21k forks source link

Use Post for Streaming Endpoints #6091

Closed NarekA closed 8 months ago

NarekA commented 8 months ago

Http streaming breaks when the input doc schema has fields which are not str, int, or float. This includes dicts, bools, and nested objects (See Issue 6090). In addition it caps the input size to 2000 characters (Much less when you factor in URL encoding)

This PR changes the get endpoints to post endpoints. Not sure if we want to keep the get endpoints for backwards compatibility.

Goals:

codecov[bot] commented 8 months ago

Codecov Report

Attention: 27 lines in your changes are missing coverage. Please review.

Comparison is base (2c3d7d3) 76.36% compared to head (ab254e5) 74.65%. Report is 1 commits behind head on master.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #6091 +/- ## ========================================== - Coverage 76.36% 74.65% -1.72% ========================================== Files 145 145 Lines 14007 14024 +17 ========================================== - Hits 10697 10469 -228 - Misses 3310 3555 +245 ``` | [Flag](https://app.codecov.io/gh/jina-ai/jina/pull/6091/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jina-ai) | Coverage Δ | | |---|---|---| | [jina](https://app.codecov.io/gh/jina-ai/jina/pull/6091/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jina-ai) | `74.65% <6.89%> (-1.72%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jina-ai#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files](https://app.codecov.io/gh/jina-ai/jina/pull/6091?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jina-ai) | Coverage Δ | | |---|---|---| | [jina/\_\_init\_\_.py](https://app.codecov.io/gh/jina-ai/jina/pull/6091?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jina-ai#diff-amluYS9fX2luaXRfXy5weQ==) | `56.00% <100.00%> (ø)` | | | [jina/clients/base/helper.py](https://app.codecov.io/gh/jina-ai/jina/pull/6091?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jina-ai#diff-amluYS9jbGllbnRzL2Jhc2UvaGVscGVyLnB5) | `79.06% <0.00%> (-6.08%)` | :arrow_down: | | [...ve/runtimes/gateway/http\_fastapi\_app\_docarrayv2.py](https://app.codecov.io/gh/jina-ai/jina/pull/6091?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jina-ai#diff-amluYS9zZXJ2ZS9ydW50aW1lcy9nYXRld2F5L2h0dHBfZmFzdGFwaV9hcHBfZG9jYXJyYXl2Mi5weQ==) | `0.00% <0.00%> (ø)` | | | [jina/serve/runtimes/worker/http\_fastapi\_app.py](https://app.codecov.io/gh/jina-ai/jina/pull/6091?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jina-ai#diff-amluYS9zZXJ2ZS9ydW50aW1lcy93b3JrZXIvaHR0cF9mYXN0YXBpX2FwcC5weQ==) | `60.16% <7.14%> (-21.99%)` | :arrow_down: | ... and [23 files with indirect coverage changes](https://app.codecov.io/gh/jina-ai/jina/pull/6091/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jina-ai)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

NarekA commented 8 months ago

Added the get endpoints back, and I will add some tests for the post endpoint. This feature will also align us with openai since they are using streaming on post endpoints too.

NarekA commented 8 months ago

@JoanFM @alaeddine-13 created a new PR that follows conventions, working on adding tests too.