search

ELS-RD / transformer-deploy

Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀
https://els-rd.github.io/transformer-deploy/
Apache License 2.0
1.64k stars 150 forks source link

Unable to optimize microsoft/deberta-v3-small model #150

Open hlkong323 opened 1 year ago

hlkong323 commented 1 year ago

Here's command and error message. Please advise and help fix it. Thanks!

sudo docker run -it --rm --gpus all -v $PWD:/project ghcr.io/els-rd/transformer-deploy:latest bash -c "cd /project && \ convert_model -m \"microsoft/deberta-v3-small\" \ --backend tensorrt onnx \ --seq-len 16 128 128 \ --task classification"

image

bminixhofer commented 1 year ago

Hi @pommedeterresautee, I have the same problem trying to optimize DeBERTa.

Do you think it's possible to fix this with a reasonable amount of effort? In my experience DeBERTa is the best encoder-style model right now so this would be super useful to have.

pommedeterresautee commented 1 year ago

Currently it s not on our todo