search

huggingface / optimum

🚀 Accelerate training and inference of 🤗 Transformers and 🤗 Diffusers with easy to use hardware optimization tools
https://huggingface.co/docs/optimum/main/
Apache License 2.0
2.54k stars 455 forks source link

Assistance exporting git-large to ONNX #874

Open gracemcgrath opened 1 year ago

gracemcgrath commented 1 year ago

Hello! I am looking to export an image captioning Hugging Face model to ONNX (specifically I was playing with the git-large model but if anyone knows of one that might be easier to deal with in terms of exporting that is great too)

I'm trying to follow these instructions for exporting an unsupported architecture, and I am a bit stuck on figuring out what base class to inherit from and how to define the custom ONNX Configuration since I'm not sure what examples to look at (the model card says this is a transformer decoder model, but it looks like i that it has both encoding and decoding so I am a bit confused)

I also found this notebook but I am again not sure if it would work with this sort of model.

Any comments, advice, or suggestions would be so helpful -- I am feeling a bit stuck with how to proceed in deploying this model in the school capstone project I'm working on. In a worst-case scenario, can I use from_pretrained in my application?

fxmarty commented 1 year ago

In a worst-case scenario, can I use from_pretrained in my application?

Yes, it is fine to use just this. Unless you need to speed up inference, make things portable, or get out of PyTorch, it's completely fine.

Ideally, a GitOnnxConfig should be added in Optimum for the export, looking at GitForCausalLM code, it seems to be a decoder-architecture usable with generate() method.

len2618187 commented 9 months ago

@gracemcgrath Hello! Have you found a solution to this problem? I am facing the same problem. Thanks