Failed to reproduce the results presented in the paper

[kisialiou]

Hello,

Your approach and results presented in the paper "CascadeTabNet: An approach for end to end table detection and structure recognition from image-based documents" impressed me a lot, so I decided to reproduce results myself using this repository.

I followed instructions and processes described in the repository and the paper, but, unfortunately, failed to obtain the same numbers.

To be more precise:

• Versions of frameworks and libraries match.

• Test set of ICDAR2019 B2-Modern (100 images) was taken from here and used for evaluation.

• Evaluation tool was taken from here.

• ICDAR 2019 model for structure recognition was taken from the official repository.

• main.py from the repository was used as an entry point

The results obtained by me differ up to 38% in case of IoU=0.9 (see attached file with results for more details).

I'd be extremely grateful if you could tell me which exact tools you've used for prediction and evaluation and suggest what I could've done wrong in my reproduction experiments.

Thank you in advance!

ICDAR2019_B2M_test.txt

[Alsonkong]

Hi @kisialiou where you get the tar.gz required for the evaluation? Thank you

[kisialiou]

Hi @Alsonkong

Here you could find the ICDAR-2019 dataset with ground truth, which is required for evaluation.

Additionally, if you're using this recommended tool for evaluation, you need to pack your predictions into tar.gz and pass it to evaluation script.