huggingface / optimum

🚀 Accelerate training and inference of 🤗 Transformers and 🤗 Diffusers with easy to use hardware optimization tools
https://huggingface.co/docs/optimum/main/
Apache License 2.0
2.49k stars 446 forks source link

onnx conversion for custom trained trocr base stage1 #964

Open Mir-Umar opened 1 year ago

Mir-Umar commented 1 year ago

Feature request

I have trained the base stage1 trocr on my custom dataset having multiline images. The trained model gives good results while using the default torch format for loading the model. But while converting the model to onnx, the model detects only first line or part of it in first line. I have used this https://github.com/huggingface/transformers/issues/19811#issuecomment-1303072202 for converting the model to onnx. Can you kindly provide the insights about what i should do differently, in order to get the desired multiline output from the onnx converted model.

Motivation

How to update the onnx conversion of trocr in order to support multiline trocr trained model

Your contribution

Trained a trocr base stage1 model for multiline dataset.

fxmarty commented 11 months ago

Hi @Mir-Umar, thank you for the report. Could you provide a reproducible example of the issue? Thank you!