Tracing CLIP-based vision models for DeepDetect

jolibrain / deepdetect

Deep Learning API and Server in C++14 support for PyTorch,TensorRT, Dlib, NCNN, Tensorflow, XGBoost and TSNE

https://www.deepdetect.com/

Other

2.52k stars 561 forks source link

Tracing CLIP-based vision models for DeepDetect #1553

Closed cchadowitz closed 8 months ago

cchadowitz commented 9 months ago

Added basic script to trace HuggingFace Transformers models based on https://huggingface.co/openai/clip-vit-large-patch14-336

Note: this does require patching transformers/models/modeling_clip.py as noted in the comments. Without the patch, the values are traced to either CPUFloatType or CUDAFloatType and so the model cannot be used on either type of device interchangeably.

Bycob commented 8 months ago

Closed because it was merged in #1554