search

microsoft / unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
https://aka.ms/GeneralAI
MIT License
19.49k stars 2.48k forks source link

Error In finetuning layoutlmv3 object detection #1310

Open vikhil0609 opened 11 months ago

vikhil0609 commented 11 months ago

I am using layoutlmv3 for Finetuning the Text , Title, Table, List, Figure from the pdf pages converted to Images. I am using Label-studio to make the dataset and exported the dataset as coco-format, But when I tried to start the training I am getting this error

AttributeError: Cannot find field 'gt_masks' in the given Instances! .

Can anyone help ? thank you

Soul-Code commented 10 months ago

same problem.

Soul-Code commented 10 months ago

I set the MASK_ON to False, and the code can run without error. But the model exhibits non-convergence and loss_box_reg_stage continues to rise with training, total_loss shows a fluctuating trend, which has been evident since the early stages of training.

Soul-Code commented 10 months ago

I think this is because the Label-studio annotation does not contain any MASK information.

SuyashManiyar commented 2 months ago

I am using layoutlmv3 for Finetuning the Text , Title, Table, List, Figure from the pdf pages converted to Images. I am using Label-studio to make the dataset and exported the dataset as coco-format, But when I tried to start the training I am getting this error

AttributeError: Cannot find field 'gt_masks' in the given Instances! .

Can anyone help ? thank you

I am using layoutlmv3 for Finetuning the Text , Title, Table, List, Figure from the pdf pages converted to Images. I am using Label-studio to make the dataset and exported the dataset as coco-format, But when I tried to start the training I am getting this error

AttributeError: Cannot find field 'gt_masks' in the given Instances! .

Can anyone help ? thank you

You can include gt_masks in your coco json annotations by either getting actual segmentation masks or making rectangular segmentation masks using bbox info