search

microsoft / table-transformer

Table Transformer (TATR) is a deep learning model for extracting tables from unstructured documents (PDFs and images). This is also the official repository for the PubTables-1M dataset and GriTS evaluation metric.
MIT License
2.01k stars 231 forks source link

Question on fine-tuning TATR with a proprietary dataset #169

Open srivatsan-sridhar99 opened 4 months ago

srivatsan-sridhar99 commented 4 months ago

Hi!

I am trying to fine tune the TATR model with a proprietary dataset. I am currently trying to convert the dataset to the same format as FinTabNet and then using the script in this repository (scripts/process_fintabnet.py) to transform that into the Pascal VOC format required by TATR.

I am then training it using the main.py file in this repo, with just one change - loading the tatr table detection (microsoft/table-transformer-detection, revision="no_timm") and tatr table structure recognition model (microsoft/table-transformer-structure-recognition-v1.1-all) from hugging face, instead of the DETR model that is built in this repository.

When I train it on the DETR model that is in this repository, I am able to train it without any issue.

The command I run to train is - python3 main.py --data_root_dir <data directory> --config_file structure_config.json

I am getting this error - tatr training error

Details on runtime env -

Would be great if anybody could help me with this!
Thanks, Srivatsan.

ali4friends71 commented 2 months ago

Hi @srivatsan-sridhar99. How did you create your custom dataset ? I have my own table images. How do I preprocess them and convert it to required format ? Any code help ? thanks In advance.