search

wenwenyu / PICK-pytorch

Code for the paper "PICK: Processing Key Information Extraction from Documents using Improved Graph Learning-Convolutional Networks" (ICPR 2020)
https://arxiv.org/abs/2004.07464
MIT License
559 stars 193 forks source link

About SROIE Dataset Preparation #54

Open ning-mz opened 3 years ago

ning-mz commented 3 years ago

Hi, that's a good work on IE, thanks.

Currently, I've tested your code on SROIE with the "document_level" setting. I used the OCR results to train and test the model, which downloaded from SROIE official website and extracted from the folder "task1&2_train(626p)" and "taks1&2_text_test(361p)". The performance looks not good as yours. May I ask how you prepared your dataset. Did you use other OCR tools to preprocess the dataset.

Thank you very much.

ning-mz commented 3 years ago

I've did some changes on original code and added some post-processing of output results. The set used 'box_and_within_box_level'. Now the performance only reached to around 88%.

Hope someone who successfully reimplemented the result in paper can give me some help, thank you.

ninjakx commented 3 years ago

@ning-mz : If you don't mind can you give some hint about post processing?

ning-mz commented 3 years ago

@ning-mz : If you don't mind can you give some hint about post processing?

Hi, basically it was depends on the output result. The code provided now is not exactly suit for the official SROIE mission.

For example, sometimes it has multiple outputs on the entity "Total", I chose the one with highest confidence mark. Meanwhile, for the output of "Address", you need to add spaces into result, cause the output of PICK is a sequence, it ignored the \n in receipts. For example:

Department of Computer Science, University of Liverpool, Liverpool

The output of PICK now will be: Department of Computer Science,University of Liverpool,Liverpool The correct result should be: Department of Computer Science, University of Liverpool, Liverpool Note the space after ,

Meanwhile, due to the original data preprocess in code, it will drop the full stop . at the end of sentence especially in "Company", like: XXX Co. Ltd. it will drop the last full stop and process it like: XXX Co. Ltd

And the official SROIE also has OCR mismatching, it will also influence the performance.

It is simple to add some rules with evaluate the results. Good luck!

ninjakx commented 3 years ago

@ning-mz : Can you help regarding this?

jorgerodriguezsj commented 3 years ago

@ning-mz

For example, sometimes it has multiple outputs on the entity "Total", I chose the one with highest confidence mark.

Please can you share how you did it? I have not managed to obtain a confidence value for each output.

Thanks

yellowjs0304 commented 2 years ago

@ning-mz @ninjakx Did you get any good way to deal this problem? (I wondered how to extract multiple data in one entity(like product price in SROIE dataset.) Could you please share some tips? @wenwenyu