Demo using FinTabNet model

[julian24bas]

I used your demo script with the pretrained PubTabNet model with no issues. However using the FinTabNet model I get following output:

python table_recognition/demo/demo.py
/opt/homebrew/Caskroom/miniconda/base/envs/MTL-TabNet/lib/python3.9/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  ../c10/core/TensorImpl.h:1156.)
  return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
Use load_from_local loader
The model and loaded state dict do not match exactly

size mismatch for decoder.cell_fc.weight: copying a param with shape torch.Size([178, 512]) from checkpoint, the shape in current model is torch.Size([281, 512]).
size mismatch for decoder.cell_fc.bias: copying a param with shape torch.Size([178]) from checkpoint, the shape in current model is torch.Size([281]).
size mismatch for decoder.embedding_cell.lut.weight: copying a param with shape torch.Size([178, 512]) from checkpoint, the shape in current model is torch.Size([281, 512]).
<thead><tr><td><b>△△<strike>△η&∩≤Аη̸∩η̸∩ААη̸∩​∩βø∩∩∩∩∩∩∩∩∩∩∩∩∩∩∩∩∩∩∩∩∩̸∩J&̸∩J&̸∩J̸∩9̸∩̸∩̸∩̸∩̸∩̸∩9∩9f∩9<sub>9fААААААV∩АVА9<sub>∩9<sub>∩9<sub>∩9<sub>∩АААААV=∩∩∩∩∩9f=αα∩∩∩∩</b></td><td><b>→E~3ʹ⇓3q∆∣∣∣‐***→<sub>◊*q≤~◊→<sub>◊▪⇑▪⇑▪⇑▪◊→¢⇑R]¢→q£'⇑<strike>qqqqqqqqqqqqqqqqqq]¢▪▪→<sub>◊→<sub>▪▪▪▪→<sub>▪▪→<sub>▪→<sub>▪→<sub>▪→<sub>''''''''''''''''''''''''σ→<sub>V~~~~~VVVV~~~~~~~~~~~~~~~~~~~~~</b></td><td><b>∆∆~~~~~~~~~~∆∆▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪∆~∆~∆~∆~∆~◊▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪~~▪~▪~~~~~~~~~~~~~~~▪V▪</b></td><td><b>cccc~~~~~~c∆∆∆∆c∆c⇑~c⇑βc⇑~~~~~~~~~~~~~~~~~~~~~~~~~~∆~∆~∆~∆~~~~~~~~!∆~~~~~!~!∆!~!c~~~~~∆!∆!∆!∆<sub>∆<sub>V∆<sub>V∆<sub>V∆<sub>V∆<sub>VVVVVVVVV<sub>VVVVVVVVVVVVVVVVVVVV~~~~~~~~~~~~~</b></td></tr><tr><td><b>∆ß∆6∆6∆∆∆6∆6χ⊕χχχ⊕ζ⊕⊕⊕⊕±±±±±ΛΛχχΛΛχ66666666⊕±±±6χ6666χχχχχχχχχχχχχχχχχχχχχχχχχχχχχχχχχχχ⊕Λχχ⊕Λχχχ⊕Λχχχχ⊕χ⊕χVVVVVVVVVχVVVχVVVχVVχVχχVχχVχχVχχVχχVχχVχχχχ</b></td><tbody><tbody><tbody></tr><tr><td><b>66666»ʹζ∆6ζ###ζ##ζ<overline>ζhζ<overline>ζ<overline>ζ<overline>ζ<overline>ζM̆66666666666666666666666666666666χ’#’666χ’####VVχVVVVVVVVχVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVχχ</b></td><td><b>√√√√←√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√</b></td><td><b>√√√√√√ΙΙΙ√√√√∙ζ√√M√M√M√M√M√√√√√√M√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√∅˂IААААААААААААА----------------</b></td><td><b>----))➢)➢c√)aaaaaaaaaaM)√I)АААaА-×jjjjjjjjI√jjI-jjI-jjI-------------------I!j!#aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-----------------</b></td></tr><tr><td><b>6666±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±6±6±±±±±6±6±6±6±6±6±66666±666±6⊕jj±6±6⊕jjj±66⊕jG⊕G±6⊕ΛVG±6±VGV±V±VVVVV±VVV±VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV</b></td><td><b>⊕</underline></underline></underline></underline></underline>*G*√√√****√√</underline>⊕j⊕⊕j⊕jjjjjjjjjjjjjjjjjjjjjj√√√√√√√√√√√√√√√√√√√√√√√√√√GπGπGπGπGπG⊕G⊕GGGGGG⊕G⊕G⊕G⊕⊕⊕⊕⊕⊕q⊕⊕⊕q⊕⊕⊕q*GVVVVVG⊕j∩VG√√√√√√√√√√√√</b></td><td><b>──f▪fjjjjj─⊕j¢√:√::::::jjD∼√√√∼jjjjjjjjjjjjjjj♦√∼∼∼√√√√√√√√√√√√√√√√√√√√√√√√√√ππππ:π:π─π─π─π─π:::::::::::::::::::::::::fπ─π─π─π─π─π─π─π─π────────────</b></td><td><b>)±±±±))➢)➢±±±#)*I)))))))±±±β±β±β±β±βjjjjjjjjjIjjjjI)±¢—±=jI#jjI)➢)N#I)➢)NI)➢)NI!j!jI!j!jI!jj!πI!j!fjaaa⊕±IIIVVVVVVIVVVVVIVVVVVVIVVVVIVVVVIVVVIVjjIVjIVj</b></td></tr><tr><td><b>⊕±±±±±±±±±±±±±*⊕±±±±±±±±±±±±±±±±±±±⩽±⩽±±±±±⩽±⩽±±±±±±±fj!*f~⩽⊕⊕⊕f⊕f⊕f⊕!V!VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV</b></td><td><b>√√</underline>√</underline></underline>√√√√√√√*√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√Ι▼√√ΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙ∩∩∩∩</b></td><td><b>√√√√ΙΙΙΙΙ√√√√ΙΙ√√√√√√√√√√√√√√√√√ΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙ!Ι!ΙΙΙΙ!Ι!Ι!Ι!Ι!Ι!ΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙ-Ι-@a</b></td><td><b>ΙΙΙΙΙΙΙΙΙΙΙΙΙ*⊕E⊕E⊕E⊕⊕β~~~~√V<overline>β~√V!βββ!ββ!β⊕!~!β<overline>▼β~√<overline>▼—<overline>▼~~~~!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!a!a⊕!!!!aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaEI</b></td></tr><tr><td><b>‐333333±±±±±±⊕±⊕±±±±±±±±±±3333±3±V3±V3▼3▼3▼±<sub>3▼±-----------------------------!-!α!V!V!V!V!V!V!VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV<overline></b></td><tbody><tbody><tbody></tr><tr><td><b>∆∆∆χ∆СΓ"χVω{ω{ωωωωωωωωωV<overline>V<overline>VVVVVVVVVV<overline>V<overline>V<overline>V<overline>▼<overline>▼<overline>▼<overline><overline><overline><overline>V<overline>V<overline>V<overline>VVVV<overline>V<overline>V<overline>V<overline>VVVVVVVVVχVVVVVVVVχVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV</b></td><td><b>—33¢—333ʹ—————←<underline>←<underline>←←←←←←←←←←←←←←jjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjφφφφφφφφφφφφφφφφφφφφφφφφ—3a<underline><underline><underline><underline><underline><underline>—3<sub>-<underline>—3<sub>—3<sub>—3<sub>-<underline>j—3E—3E—3E—3E&EEEEEEE-----</b></td><td><b>—∆∆▪▪▪—3———∆▪———aaaaMMMMMMjjjjjjjjjjjjjjjjjjjjjjjjjjj——------------̸jjjjjj̸jjj▪—-------̸-̸---̸-------999999999999999999999999999999999-----------------</b></td><td><b>—∆-±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±-—±-—±-—±-±-—±-±-±-—±-9φφφφφφφφφφ<sub>~—±-9φφφφφφφ<sub><sub><sub><sub><sub><sub><sub><sub><sub><sub><sub><sub><sub>—±<sub>±<sub>±<sub>±<sub>±<sub>±-±<sub>-9<sub>-9—±-9<sub>-9—<sub>---------------</b></td></tr><tr><td><b>∆∆∆∆JJJJ∆∆∆∆∆∆6⊕J∆∆∆∆∆6⊕±JJVJJJJVVJJJJJJJJJ∆6∆6∆6∆6∆6∆6∆6∆6∆6∆6∆6J∆6∆6JJJJJJJ:JJJJ:JJJJJJJJJJJJJJJJJJJVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVJVVJVVVVVVJVJV</b></td><td><b>⊕</underline></underline></underline></underline>¢</underline>J¢</underline>⊕j*<underline><underline><underline><underline><underline></underline>⊕</underline>⊕</underline>⊕​j±J:</underline></underline></underline>±Jjjjjjjjjjj¢↔⊕j¢↔⊕j↔⊕↔⊕j¢6⊕j¢↔⊕j¢G<underline></underline>⊕j¢G<underline><underline>G<underline>G⊕j¢G<underline><underline></underline>G<underline></underline>⊕GGG⊕GJ¢GJGJyGJ:¢GJ:¢GJ:J:±J:±±±GJyGJ⊕j</underline>⊕jGEσ√±J√±σ√σ√EGE</underline></underline></underline></underline></b></td><td><b>────~¢J───D√±D¢─D::::MJ:MJ:J:──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────</b></td><td><b>∆∆∆⇑¢±∆∆∆±±±±*∆∆∆∆±±±±±±±±±​ł∆​jjDδ±D¢∆∆∆∆∆∆∆∆∆∆∆∆∆∆∆∆∆∆∆∆∆∆„¢∆„¢∆¢±J:¢∆÷¢J¢VJ¢V∆∆∆∆∆VJ¢V∆∆∆∆∆∆∆∆V±V::::—:::VVVVVVVVVVVVVVVVVVVVVVVVVVVVVJVJVJVJVJVJEVE</b></td></tr><tr><tbody><td><b>√√√√←←←q<underline>←√√√<underline><underline><underline><underline><underline>←<underline>q<underline><underline>←<underline>q<underline>←√√√√√<underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline><underline>''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''<underline><underline><underline><underline><underline><underline></b></td><td><b>√V√~√±~√√√√√√<underline><underline><underline><underline><underline><underline><underline><underline><underline>~√M√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√√V<overline>VVVV<underline><underline><underline><underline><underline><underline><underline><underline><underline>~VVVVVVVVVVVVV~√V<overline>√V~√√√√√√√√√</b></td><td><b>ΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙ±±±±±±±±±±±±±±±±±±±±±±±±±±±±<overline>ΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙΙ±±±±V±V±V±V±±±±V±V±V±V±V±V±V±V~~~~~~~~~~~~~~~~</b></td></tr></thead></tr></tbody>

[AmineMsr]

i get the same output as well with the fintabnet model, with some encoding errors sometimes as shown below, did you find any solutions to that or this just what the model gives and there is nothing that can be done about it

Traceback (most recent call last):
  File "demo.py", line 412, in <module>
    f.write(pred_html + '\n')
  File "C:\Users\amine\anaconda3\envs\myenv\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u0301' in position 338: character maps to <undefined>

[julian24bas]

If I remember correctly changing the config argument in line 339 of the demo.py script to ./configs/textrecog/master/table_master_ResnetExtract_Ranger_0705_FinTabNet_cell150_batch4.py helped getting rid of the error. The result was still worse than with the PubTabNet model.

[lerndeep]

I also test and the result is not good. @namtuanly could you please let us know?

[namtuanly]

Hi all, The demo file for FinTabNet is different from PubtabNet. I have committed the demo file for FinTabNet (demo_FinTabNet.py). Could you try it again?

[julian24bas]

Hi, thanks for your effort!

I just tried and got two smaller errors:

1. File "/MTL-TabNet/table_recognition/demo/demo_FinTabNet.py", line 343, in <module>
   tablemaster_inference.print_num_params()
   AttributeError: 'Structure_Recognition' object has no attribute 'print_num_params'

   I just removed that line as it seems to be for debugging

2. deal_bb()function was missing an argument. I figured this might fix it?

   def deal_bb(result_token, search_token):
   """
   In our opinion, <b></b> always occurs in <thead></thead> text's context.
   This function will find out all tokens in <thead></thead> and insert <b></b> by manual.
   :param result_token:
   :return:
   """
   # find out <thead></thead> parts.
   thead_pattern = f'<{search_token}>(.*?)</{search_token}>'

The result for the sample image are better now:

[artshcherbina]

Hello, @julian24bas . Unfortunately, you image is not visible. I've got the following image using FinTabNet/epoch_17.pth:

@namtuanly, Thank you for your work! We need vertical flip for FinTabNet dataset, because generated labels are vertically flipped.