jina-ai / clip-as-service

πŸ„ Scalable embedding, reasoning, ranking for images and sentences with CLIP
https://clip-as-service.jina.ai
Other
12.48k stars 2.07k forks source link

feat: support custom onnx file and update model signatures #761

Closed ZiniuYu closed 2 years ago

ZiniuYu commented 2 years ago

This PR allows the user to set filepath for pretrained custom onnx model and unify the signature of tensorrt, onnx, and huggingface clip runtime. We also add md5 verification to make sure users download the latest and correct models. Since tensorrt models are extremely picky for many dependencies, we no longer support tensorrt model hosting, which means that it will first download the latest onnx model and convert to trt during runtime and save for it later use.

TODO:

codecov[bot] commented 2 years ago

Codecov Report

Merging #761 (94008ac) into main (ed1b92d) will increase coverage by 0.71%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #761      +/-   ##
==========================================
+ Coverage   78.31%   79.03%   +0.71%     
==========================================
  Files          17       17              
  Lines        1213     1240      +27     
==========================================
+ Hits          950      980      +30     
+ Misses        263      260       -3     
Flag Coverage Ξ”
cas 79.03% <100.00%> (+0.71%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Ξ”
server/clip_server/executors/clip_torch.py 87.03% <ΓΈ> (ΓΈ)
server/clip_server/executors/clip_onnx.py 85.48% <100.00%> (ΓΈ)
server/clip_server/executors/helper.py 100.00% <100.00%> (ΓΈ)
server/clip_server/model/clip.py 65.60% <100.00%> (+6.31%) :arrow_up:
server/clip_server/model/clip_onnx.py 96.96% <100.00%> (+0.96%) :arrow_up:
server/clip_server/model/clip_trt.py 92.10% <100.00%> (-0.21%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Ξ” = absolute <relative> (impact), ΓΈ = not affected, ? = missing data Powered by Codecov. Last update ed1b92d...94008ac. Read the comment docs.

github-actions[bot] commented 2 years ago

:memo: Docs are deployed on https://ft-improve_onnx--jina-docs.netlify.app :tada: