search

aws-neuron / aws-neuron-samples

Example code for AWS Neuron SDK developers building inference and training applications
Other
101 stars 32 forks source link

Support for TR-OCR Base Printed for AWS Neuron(Inferentia) #3

Closed umanniyaz closed 1 year ago

umanniyaz commented 1 year ago

Please provide support for Tr-OCR Base Printed conversion using Neuron and its inference in jit trace , getting this error when trying through the uploaded notebook, the process flow is as below after meeting tensor shape as 768, encoder complies but decoder fails to compile with neuron command -9 @hyandell @mattmcclean @aws-maens @brunopistone

image

image

image

image

awsilya commented 1 year ago

Hi @umanniyaz I opened a ticket with our team. Will update you when we have more information.
Thank you.

aws-rishyraj commented 1 year ago

Hi @umanniyaz, I've looked into the issue and I've determined that it's a size mismatch issue with the decoder when passing in the last_hidden_states from the encoder. To fix this, please change the size of encoder_hidden_states to torch.rand([1,577,768]).

If you want to experiment with other sized VisionEncoderDecoder models I suggest checking the input and output sizes of the models. This can be verified by running the non-traced model and checking the shape with .shape.

Hope this helps!

aws-rishyraj commented 1 year ago

@umanniyaz I've reopened the issue, so I can verify that the proposed fix works on your end. Please verify on your end when you get the chance to.

aws-taylor commented 1 year ago

Hello @umanniyaz,

We haven't heard from you in a while, so I'm going to go ahead and resolve this. Please feel free to re-open if the proposed fix does not work for you.

-T