search

DevashishPrasad / CascadeTabNet

This repository contains the code and implementation details of the CascadeTabNet paper "CascadeTabNet: An approach for end to end table detection and structure recognition from image-based documents"
MIT License
1.46k stars 422 forks source link

Added two new classes(rows and columns) to data, but model is able to train only for column, not learning rows. #175

Open annusrcm opened 1 year ago

annusrcm commented 1 year ago

The original base model is trained for bordered, borderless and cells. If I add my custom invoice data with labelled data for these three classes it works very well, I am able to extend the model for my data. However if I add two more classes(rows and columns) to data model is able to train only for column, it is not learning rows.

borderless, row, col and bordered ----> If I train for these classes, it is able to learn column(upto 98% confidence) and tables(upto 98% confidence) well, but it fails for rows. The score for row does not go more than 20%. Here is an example for row prediction: {'l': 0, 't': 3508, 'r': 83, 'b': 3508, 'confidenceScore': '19.895745813846588', 'class': 'row'}.

For tables and columns : {'l': 1616, 't': 1333, 'r': 1936, 'b': 1571, 'confidenceScore': '99.99583959579468', 'class': 'column'} {'l': 248, 't': 1337, 'r': 664, 'b': 1544, 'confidenceScore': '99.9913215637207', 'class': 'column'} {'l': 2047, 't': 1331, 'r': 2414, 'b': 1567, 'confidenceScore': '99.99083280563354', 'class': 'column'} {'l': 129, 't': 1343, 'r': 260, 'b': 1530, 'confidenceScore': '99.45705533027649', 'class': 'column'} {'l': 0, 't': 3508, 'r': 83, 'b': 3508, 'confidenceScore': '19.89872008562088', 'class': 'column'} {'l': 95, 't': 1342, 'r': 2471, 'b': 1537, 'confidenceScore': '99.66540336608887', 'class': 'borderless'}

I am smudging the image and training, using mmdetection v2. Has anyone else faced similar problem. If more info/data is needed to help understand this, please let me know I will add.

linkstatic12 commented 12 months ago

How did u make it run? Which version of pytorch, cuda are you using? Which version of python are you using?

linkstatic12 commented 12 months ago

are you using windows or linux?

annusrcm commented 11 months ago

  1. torch==1.7.0+cu110 torchvision==0.8.1+cu110 torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

  2. mmcv-full==1.4.0 -f https://download.openmmlab.com/mmcv/dist/cu110/torch1.7.0/index.html

  3. I am using ubuntu.

  4. Made changes in config file to accomodate 4 classes

  5. After create following data folder coco trainval.json test.jsom VOC2007 Annotations : this contains xmls JPEGImages : This contains smudge images ImageSets Main text.txt trainval.txt

  6. I ran using this command: python tools/train.py

linkstatic12 commented 11 months ago

I know this is really unorthodox but is it possible that we can do an online meeting? I have been stuck on this problem for the past 2 months now and I am unable to solve it. I would be grateful if you can show me the changes.

linkstatic12 commented 11 months ago

ImportError: cannot import name 'deform_conv_cuda' from 'mmdet.ops.dcn' I get this error

AGRocky commented 11 months ago

Hi Folks, greetings of the day to y'all. I have successfully tried to run the code in the notebook with adding few other functionality to it and it works fantastic on colab. However, now I want this to run on my local machine with VScode IDE as modules. Please guide me through this. It would be deeply appreciated.

linkstatic12 commented 11 months ago

Can you please share the Colab link?

annusrcm commented 11 months ago

I know this is really unorthodox but is it possible that we can do an online meeting? I have been stuck on this problem for the past 2 months now and I am unable to solve it. I would be grateful if you can show me the changes.

Once we get it working, I will post the soultion here. We are still stuck.