What fraction of the test set did the model achieve perfect table (structure) recognition on?

microsoft / table-transformer

Table Transformer (TATR) is a deep learning model for extracting tables from unstructured documents (PDFs and images). This is also the official repository for the PubTables-1M dataset and GriTS evaluation metric.

MIT License

2.31k stars 257 forks source link

What fraction of the test set did the model achieve perfect table (structure) recognition on? #13

Closed MrinalJain17 closed 2 years ago

MrinalJain17 commented 2 years ago

The GriTS score provides performance estimate at the cell level. I was curious about how many of the images in the test set did the model get the table EXACTLY right.

@rohithpv @bsmock

bsmock commented 2 years ago

Hi, in the current version of the paper this is reported as Accuracy (AccCont) in Table 4. According to this definition of exact recognition (the text content at every grid location matches the ground truth exactly), the percentage of tables recognized exactly by the baseline DETR model in our paper is 81.38% overall, 94.68% for simple tables, and 69.44% for complex tables.

Hope that helps!

Best, Brandon

MrinalJain17 commented 2 years ago

Thanks @bsmock