microsoft / unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
https://aka.ms/GeneralAI
MIT License
19.62k stars 2.5k forks source link

Questions about loss of layoutlmv3 when finetuning on custom datasets #1536

Open hengrui0516 opened 5 months ago

hengrui0516 commented 5 months ago

Hi, thanks for your excellent work! I add some other documents and their annotations into publaynet to create a new custom dataset. I try finetuning Layoutlmv3 on it. But these other documents only have bbox annotations but lack of segmentation(polygon) information. I wonder if it matters, And I wonder whether bbox loss and polygon loss will be added together or work independently.