search

jpWang / LiLT

Official PyTorch implementation of LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding (ACL 2022)
MIT License
335 stars 40 forks source link

not able to reproduce results #4

Closed sujit420 closed 2 years ago

sujit420 commented 2 years ago

First of all, thanks for this nice work. However, I am unable to reproduce the results given in the table. I was training Funsd for relation extraction task. Please let me know if I am doing anything wrong. python examples/run_xfun_re.py --model_name_or_path lilt-infoxlm-base --tokenizer_name xlm-roberta-base --output_dir ls_re_xfund_lilt-infoxlm-base --do_train --do_eval --lang en --max_steps 20000 --per_device_train_batch_size 2 --warmup_ratio 0.1 I am getting eval f1 of 0.47

jpWang commented 2 years ago

Hi, thanks for your feedback. Since my GPUs are being occupied by other experiments, I will soon check the performance of RE task in a few days.

jpWang commented 2 years ago

@sujit420 The fine-tune strategy of RE task has been updated. You can try the newest codes and command lines.

sujit420 commented 2 years ago

@sujit420 The fine-tune strategy of RE task has been updated. You can try the newest codes and command lines.

Thanks, @jpWang for the quick response. Though I am yet to get the reported score of 0.6276 in RE for funsd data. But I am able to get 0.606 eval f1 which is reasonable. Also, I would like to know the process to get results reported for multi-task fine-tuning on funsd/xfund? Thanks in advance.

jpWang commented 2 years ago

For multi-task fine-tuning, you can simply expand max_steps by as many times as the number of languages (=8 for FUNSD+XFUND). You can refer to https://github.com/jpWang/LiLT#multi-task-relation-extraction-on-xfund.

vibeeshan025 commented 2 years ago

Hi, Both of you guys, how long it took you to run this process, what's the setup used.