search

NormXU / ERNIE-Layout-Pytorch

An unofficial Pytorch implementation of ERNIE-Layout which is originally released through PaddleNLP.
http://arxiv.org/abs/2210.06155
MIT License
96 stars 11 forks source link

DocVQA fine-tuning example? #9

Closed JP-Leite closed 1 year ago

JP-Leite commented 1 year ago

Are there any examples of how we can fine tune with the DocVQA dataset?

NormXU commented 1 year ago

Here is a simple example of applying ERNIE on the VQA task.

This is a complete pipeline of fine-tuning on DocVQA that I strongly recommend you read. You can simply replace the tokenizer and model with ERNIE I provided and try to train a model for your own task.