search

jina-ai / clip-as-service

🏄 Scalable embedding, reasoning, ranking for images and sentences with CLIP
https://clip-as-service.jina.ai
Other
12.38k stars 2.06k forks source link

TensorRT support for openai/clip-vit-large-patch14-336 #914

Open junwang-wish opened 1 year ago

junwang-wish commented 1 year ago

Is there a fundamental technical limitation on not being able to support TensorRT for openai/clip-vit-large-patch14-336 ? Just wanna understand why most 768-dim embedding models are not supported according to https://clip-as-service.jina.ai/user-guides/server/#model-support

ZiniuYu commented 1 year ago

Thank you for your interest in clip-as-service and for reaching out with your question.

To answer your question, we want to clarify that clip-as-service aims to support as many models as possible, including the openai/clip-vit-large-patch14-336 model. However, due to the complex compatibility issues between TensorRT and other packages, such as hardware and CUDA version requirements, we are currently unable to support TensorRT for this model.

We appreciate your feedback and interest in TensorRT support, but at this moment, we don't have any concrete plans to add it in the near future due to our current capacity. However, we encourage you to continue exploring and experimenting with different solutions, and if you find a way to make TensorRT work with this model, please feel free to share it with the community so that others can benefit from your findings.

Thank you for your support and contribution to the open source community.