search

jina-ai / jina

☁️ Build multimodal AI applications with cloud-native stack
https://docs.jina.ai
Apache License 2.0
20.87k stars 2.22k forks source link

Indexing with examples (crossmodal) halts due to insufficient resources #2202

Closed rutujasurve94 closed 3 years ago

rutujasurve94 commented 3 years ago

Indexing using: python app.py -t index -n 100 -s 2 -d 'f8k' -m clip for cross modal search example with 12 GB RAM allocated for Docker and 6 cores. The indexing hangs after the 11th IndexRequest from Gateway.

Screenshots

Screenshot 2021-03-18 at 12 13 17 PM
Huffon commented 3 years ago

Same issue. But is this related to computation resources?

FionnD commented 3 years ago

@bwanglzu to try and reproduce

FionnD commented 3 years ago

Hi @bwanglzu Did you manage to do this?

bwanglzu commented 3 years ago

working on this today @FionnD

bwanglzu commented 3 years ago

lower down the request_size parameter with jina should resolve the issue. While during debugging, figured out another bug, after 1.1.7 release. (in imageencoder)

description:

 <module>\nRuntimeError: 
Given groups=1, weight of size [768, 3, 32, 32], 
expected input[6, 224, 224, 3] to have 3 channels, 
but got 224 channels instead\n')

expected input is num_samples, image_width, image_height, num_channels, we get num_samples, num_channels, image_width, image_height.

if you have such encode issue, please downgrade jina to 1.1.5

FionnD commented 3 years ago

Is there a PR for that bug?

FionnD commented 3 years ago

For the request size issue, can you open a PR with the same troubleshooting tips, please? . Then we can close this issue.