search

microsoft / unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
https://aka.ms/GeneralAI
MIT License
20.2k stars 2.55k forks source link

Using LayoutLM for Relation Extraction on FUNSD #394

Closed RishabhMaheshwary closed 3 years ago

RishabhMaheshwary commented 3 years ago

Question Is there an implementation of LayoutLM for relation extraction on FUNSD? If not, what are the steps I should follow to perform relation extraction with LayoutLM.

wolfshow commented 3 years ago

@RishabhMaheshwary please refer to the code at https://github.com/microsoft/unilm/tree/master/layoutxlm

RishabhMaheshwary commented 3 years ago

@wolfshow the link https://github.com/microsoft/unilm/tree/master/layoutxlm is for finetuning on XFUND dataset. XFUND and FUNSD has different formats. Is there a script to convert or load FUNSD in XFUND format?

RishabhMaheshwary commented 3 years ago

Also, for the LayoutLM model only the TokenClassifiction head is available while fine-tuning, the relation extraction head is not available for LayoutLM as it is for LayoutLMv2 or LayoutXLM.

munish0838 commented 1 year ago

Also, for the LayoutLM model only the TokenClassifiction head is available while fine-tuning, the relation extraction head is not available for LayoutLM as it is for LayoutLMv2 or LayoutXLM.

Hi @RishabhMaheshwary, could you get through the issue and convert FUNSD to XFUND format?

RishabhMaheshwary commented 1 year ago

Hi, No I was unable to that. Maybe you can convert relations to question answer pairs and finetune LayoutLM on FUNSD using this huggingface model.

munish0838 commented 1 year ago

Hi, No I was unable to that. Maybe you can convert relations to question answer pairs and finetune LayoutLM on FUNSD using this huggingface model.

I found this related issue, might be helpful https://github.com/microsoft/unilm/issues/586

Muhammad-Hamza-Jadoon commented 3 months ago

Hi, No I was unable to that. Maybe you can convert relations to question answer pairs and finetune LayoutLM on FUNSD using this huggingface model.

I found this related issue, might be helpful #586

Hi munish. were you able to figure out the differences in funsd and xfund datasets? i am trying to finetune funsd for relation extraction