search

huggingface / optimum

🚀 Accelerate training and inference of 🤗 Transformers and 🤗 Diffusers with easy to use hardware optimization tools
https://huggingface.co/docs/optimum/main/
Apache License 2.0
2.5k stars 448 forks source link

BetterTransformer for florence2 #1995

Open ksooklall opened 1 month ago

ksooklall commented 1 month ago

Feature request

NotImplementedError: The model type florence2 is not yet supported to be used with BetterTransformer. Feel free to open an issue at https://github.com/huggingface/optimum/issues if you would like this model type to be supported.

Motivation

Florence-2 is a small (compared to other vLLM) and widely used model thus a speed up in inference will be very helpful for everyone

Your contribution

I can help with a PR but I would need some guidance.

kinchahoy commented 2 weeks ago

+1 to offering to help and needing this support.

lieding commented 2 weeks ago

+1 to offering to support export own fine-tuned version