search

DevashishPrasad / CascadeTabNet

This repository contains the code and implementation details of the CascadeTabNet paper "CascadeTabNet: An approach for end to end table detection and structure recognition from image-based documents"
MIT License
1.5k stars 430 forks source link

Added two new classes(rows and columns) to data, but model is able to train only for column, not learning rows. #175

Open annusrcm opened 1 year ago

annusrcm commented 1 year ago

The original base model is trained for bordered, borderless and cells. If I add my custom invoice data with labelled data for these three classes it works very well, I am able to extend the model for my data. However if I add two more classes(rows and columns) to data model is able to train only for column, it is not learning rows.

borderless, row, col and bordered ----> If I train for these classes, it is able to learn column(upto 98% confidence) and tables(upto 98% confidence) well, but it fails for rows. The score for row does not go more than 20%. Here is an example for row prediction: {'l': 0, 't': 3508, 'r': 83, 'b': 3508, 'confidenceScore': '19.895745813846588', 'class': 'row'}.

For tables and columns : {'l': 1616, 't': 1333, 'r': 1936, 'b': 1571, 'confidenceScore': '99.99583959579468', 'class': 'column'} {'l': 248, 't': 1337, 'r': 664, 'b': 1544, 'confidenceScore': '99.9913215637207', 'class': 'column'} {'l': 2047, 't': 1331, 'r': 2414, 'b': 1567, 'confidenceScore': '99.99083280563354', 'class': 'column'} {'l': 129, 't': 1343, 'r': 260, 'b': 1530, 'confidenceScore': '99.45705533027649', 'class': 'column'} {'l': 0, 't': 3508, 'r': 83, 'b': 3508, 'confidenceScore': '19.89872008562088', 'class': 'column'} {'l': 95, 't': 1342, 'r': 2471, 'b': 1537, 'confidenceScore': '99.66540336608887', 'class': 'borderless'}

I am smudging the image and training, using mmdetection v2. Has anyone else faced similar problem. If more info/data is needed to help understand this, please let me know I will add.

linkstatic12 commented 1 year ago

How did u make it run? Which version of pytorch, cuda are you using? Which version of python are you using?

linkstatic12 commented 1 year ago

are you using windows or linux?

annusrcm commented 1 year ago

  1. torch==1.7.0+cu110 torchvision==0.8.1+cu110 torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

  2. mmcv-full==1.4.0 -f https://download.openmmlab.com/mmcv/dist/cu110/torch1.7.0/index.html

  3. I am using ubuntu.

  4. Made changes in config file to accomodate 4 classes

  5. After create following data folder coco trainval.json test.jsom VOC2007 Annotations : this contains xmls JPEGImages : This contains smudge images ImageSets Main text.txt trainval.txt

  6. I ran using this command: python tools/train.py

linkstatic12 commented 1 year ago

I know this is really unorthodox but is it possible that we can do an online meeting? I have been stuck on this problem for the past 2 months now and I am unable to solve it. I would be grateful if you can show me the changes.

linkstatic12 commented 1 year ago

ImportError: cannot import name 'deform_conv_cuda' from 'mmdet.ops.dcn' I get this error

AGRocky commented 1 year ago

Hi Folks, greetings of the day to y'all. I have successfully tried to run the code in the notebook with adding few other functionality to it and it works fantastic on colab. However, now I want this to run on my local machine with VScode IDE as modules. Please guide me through this. It would be deeply appreciated.

linkstatic12 commented 1 year ago

Can you please share the Colab link?

annusrcm commented 1 year ago

I know this is really unorthodox but is it possible that we can do an online meeting? I have been stuck on this problem for the past 2 months now and I am unable to solve it. I would be grateful if you can show me the changes.

Once we get it working, I will post the soultion here. We are still stuck.