search

clovaai / donut

Official Implementation of OCR-free Document Understanding Transformer (Donut) and Synthetic Document Generator (SynthDoG), ECCV 2022
https://arxiv.org/abs/2111.15664
MIT License
5.52k stars 443 forks source link

Donut generate ONLY <s><s>...<s></s> #303

Open SerenaTetart opened 1 month ago

SerenaTetart commented 1 month ago

After training the model, I have a normal training and validation loss decreasing over each epoch, only to return during generation the same token: text_sequence: <s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s>...<s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s></s>

Edit: I solved it by using transformer version 4.40.2

svchostdotexe commented 1 month ago

that solved it for you? i have the same issue, but when i upgrade to 4.40.2 i get the following error: TypeError: prepare_inputs_for_inference() got an unexpected keyword argument 'cache_position'

could you be so kind and post all your installed lib versions? (pip list and conda list)

edit: introducing the "cache_position" as an argument fixes the issue, now i can train the model with transformer==4.40.2! sadly that does not fix the <s><s><s><s><s><s> issue for me. any help would be greatly appreciated!

DriraYosr commented 1 month ago

I had the same issue with the version of transformers 4.38.2. Dowgrading transformers to the version 4.25.1 solved the issue!

SerenaTetart commented 1 month ago

@svchostdotexe the 2nd reason why you're getting this error might be because you have too much text or token in your training set

nromero125 commented 1 week ago

Im having the same issue, I tried with multiple versions of transformers and result in the same. I was following this article https://medium.com/@paul_lefevre/information-extraction-from-id-documents-with-donut-792ca80d2a93