search

DevashishPrasad / CascadeTabNet

This repository contains the code and implementation details of the CascadeTabNet paper "CascadeTabNet: An approach for end to end table detection and structure recognition from image-based documents"
MIT License
1.49k stars 427 forks source link

Discussion and Query: I was able to convert the model to the latest build of mmdet #77

Closed NISH1001 closed 4 years ago

NISH1001 commented 4 years ago

Hi @DevashishPrasad and the team,

First of all, thanks a lot for the paper.

I went through the paper. It's amazing that the pre-trained model is able to detect the table seamlessly. I was able to convert the model to the latest mmdet builds. It's working fine for the CPU (also works well in GPU...). The predictions seem the same. So, cheers! Kinda took many hours to figure out how to do it... :) Although, it's inevitable that inference time would bump up drastically.

(Side note: I am still trying to put efforts on replicating the benchmarks with the converted model. I guess I am procrastinating on that...haha)

Secondly, I have a query regarding architecture.

(A naive query...sorry if I sound stupid...). Is there a way to modify the architecture to predict the table structures? Say...using custom features from a document graph...adding parent information to cells?

Lastly, are there any rooms for optimizing the training?

Like adding (hard) negative samples. The model gives some weird False Positives like detecting faces as cells. Haha... I presume...if we could add some other textual embeddings...will it perform better? (Although we could do some post-processing for FPs)

Here's one sample: image

ahmed-friendlyhealth commented 4 years ago

@NISH1001 It would be helpful for everyone if you can share the model and the config file that works with mmdet 2.* Thanks

luke4u commented 4 years ago

@NISH1001 , +1, would be very nice if you can share your models and config. Thank you!

YoshitakaOyama commented 4 years ago

@NISH1001 , I would be very happy if you could share the model and config file. Thank you!

NISH1001 commented 4 years ago

Hi folks, Sorry been busy with personal works. I'd try to share the conversion script this weekend.

I did the conversion in 2 stages:

Once the final model is available, we could directly use the config from mmdet 2.3.

(Note: The official script from mmdet doesn't work in colab because of the weird path problem. So, you'd have to change a few lines.)

anshulp2912 commented 4 years ago

@NISH1001 If possible make a colab for the community. That would help a lot since the tutorial is too confusing! Kudos to your efforts!

iiLaurens commented 4 years ago

Indeed if you could share the conversion script or preferably even the snapshot files, it would be greatly appreciated! A month ago I spent significant time to convert the models myself, but as you already mentioned you run into weird path problems. Even after modifying the conversion file manually I wasn't able successfully load the converted snapshots into mmdet 2.x. Would be interested to see how you solved all of this.

iiLaurens commented 4 years ago

I was also able to convert the model to mmdet v2.x. I posted details in #71

NISH1001 commented 4 years ago

@iiLaurens thanks for that. I am kinda busy with works. So I am not able to clean up my notebooks and other things (otherwise posting without any clean up would have confused everyone)... Anyway, glad that the conversion is working well with other people... If the converted model is working well from your script, I'll close this issue. Thanks! (Still waiting for the queries to be answered...I guess everyone's kinda busy...) :)