search

jina-ai / jina

☁️ Build multimodal AI applications with cloud-native stack
https://docs.jina.ai
Apache License 2.0
20.87k stars 2.22k forks source link

fix: make gateway load balancer should stream results #6024

Closed alaeddine-13 closed 1 year ago

alaeddine-13 commented 1 year ago

closes: https://github.com/jina-ai/jina/issues/6020

Prior to this request, a Gateway Load balancing server will send an incoming request to the executor and wait until it returns all chunks before sending them back to the client. After this fix, the gateway will rather stream chunks as soon as they come to the gateway

codecov[bot] commented 1 year ago

Codecov Report

Merging #6024 (8c16a2e) into master (36e67f5) will decrease coverage by 3.20%. The diff coverage is 85.71%.

@@            Coverage Diff             @@
##           master    #6024      +/-   ##
==========================================
- Coverage   77.63%   74.44%   -3.20%     
==========================================
  Files         144      142       -2     
  Lines       13784    13742      -42     
==========================================
- Hits        10701    10230     -471     
- Misses       3083     3512     +429
Flag Coverage Δ
jina 74.44% <85.71%> (-3.20%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
jina/serve/runtimes/gateway/request_handling.py 80.55% <85.71%> (-9.45%) :arrow_down:

... and 33 files with indirect coverage changes

alaeddine-13 commented 1 year ago

@JoanFM I added test cases that assert the type write effect is preserved (docs are streamed as soon as they come in to the gateway). The one with protocol=http and include_gateway=True fails if the fix commit is reverted