any idea on Donut model for Key-value extraction?

NielsRogge / Transformers-Tutorials

This repository contains demos I made with the Transformers library by HuggingFace.

MIT License

8.48k stars 1.33k forks source link

any idea on Donut model for Key-value extraction? #308

Open yang0369 opened 1 year ago

yang0369 commented 1 year ago

just happened to see that Donut model is good at form understanding task and under MIT licence, so I am wondering anyone has tried using Donut to extract key-value pairs?

I checked Niels' tutorials and did not find the corresponding finetuning notebook, maybe the most similar one is Donut finetuning on CORD dataset, which is not exactly the same. so I believe some downstream modification works are needed here.

NielsRogge commented 1 year ago

Hi,

CORD is doing key-value extraction for receipts. You can just define your own custom dataset.

yang0369 commented 1 year ago

Hi,

CORD is doing key-value extraction for receipts. You can just define your own custom dataset.

Thanks for the info. I have briefly looked through the notebook, seems the CORD notebook for key-value requires a list of pre-defined keys(which means the keys are fixed), is there a way to extract random key-values like what LayoutXLM SER + RE does?