search

jina-ai / clip-as-service

🏄 Scalable embedding, reasoning, ranking for images and sentences with CLIP
https://clip-as-service.jina.ai
Other
12.48k stars 2.07k forks source link

Does the “ViT-L/14” support conversion to TensorRT parameter files? #953

Open gyd-a opened 1 month ago

gyd-a commented 1 month ago

Code file name "clip_txt.py": _MODELS = [ 'RN50::openai', 'RN50::yfcc15m', 'RN50::cc12m', 'RN101::openai', 'RN101::yfcc15m', 'RN50x4::openai', 'ViT-B-32::openai', 'ViT-B-32::laion2b_e16', 'ViT-B-32::laion400m_e31', 'ViT-B-32::laion400m_e32', 'ViT-B-16::openai', 'ViT-B-16::laion400m_e31', 'ViT-B-16::laion400m_e32',

older version name format

'RN50',
'RN101',
'RN50x4',
# 'RN50x16',
# 'RN50x64',
'ViT-B/32',
'ViT-B/16',
# 'ViT-L/14',
# 'ViT-L/14@336px',

]

'ViT-L/14' is commented out. Whether it can be opened and converte "ViT-L/14" model file.

gyd-a commented 1 month ago

https://forums.developer.nvidia.com/t/tensorrt-inference-api-that-open-clip-vit-l-14-is-slowing-down/309551/3 My successful convertion. "ViT-L/14" ---> .onnx ---> .trt But inference with the tensorrt framework is slower. Is this a normal phenomenon? The link above has some details.