search

jina-ai / clip-as-service

🏄 Scalable embedding, reasoning, ranking for images and sentences with CLIP
https://clip-as-service.jina.ai
Other
12.48k stars 2.07k forks source link

TensorRT for large models #805

Closed RaiAmanRai closed 2 years ago

RaiAmanRai commented 2 years ago

Hi, are there any particular reasons why the TensorRT models for large architecture like ViT-L-14 and ViT-L-14@336@px are not supported?

I tried to convert onnx model for ViT-L-14@336px into trt myself, but could't due to an unsupported layer.

Any thoughts?

ZiniuYu commented 2 years ago

Hi @RaiAmanRai , thank you for asking! As you have mentioned, there are some unsupported operation issues and other things such as size limits, hardware compatibilities that need to be addressed before we can fully support those models. We are investigating those blockers and will support them in the future releases if possible. You are more than welcome to share any observations or solutions! Cheers 🍻

RaiAmanRai commented 2 years ago

Cheers and thanks for quick replies. Will surely share if I find anything unusual.