clovaai / spade

Apache License 2.0
81 stars 20 forks source link

SPADE

Introduction

Task

img_1.png

Setup

  1. conda create --name spade python==3.7.10

  2. conda activate spade

  3. git clone [this-repo]

  4. pip install -r requirements

  5. Download data.tar.gz from here. The file also include the small model trained on CORD dataset.

    mv data.tar.gz [project-dir]
    tar xvfz data.tar.gz
  6. Download pretrained multi-lingual bert

cd scripts
python download_pretrained_models.py
  1. Test the code with the sample data (input: ./data/samples/cord_predict.json)

    bash scripts/predict_cord.sh

  2. (Optional) Download funsd dataset

    bash scripts/preprocess_funsd.sh

Data

Input (type1)

Test output

CORD

{
  "test__avg_loss": 0.08372728526592255,
  "test__f1": 0.9101991060544494,
  "test__precision_edge_avg": 0.932888658103816,
  "test__recall_edge_avg": 0.9192414351541544,
  "test__f1_edge_avg": 0.9259259429039002,
  "test__precision_edge_of_type_0": 0.9672624647224836,
  "test__recall_edge_of_type_0": 0.9710993577635059,
  "test__f1_edge_of_type_0": 0.9691771137713262,
  "test__precision_edge_of_type_1": 0.8985148514851485,
  "test__recall_edge_of_type_1": 0.8673835125448028,
  "test__f1_edge_of_type_1": 0.882674772036474
}

FUNSD

In addition to the scores shown in CORD example, it includes

{
  "p_r_f1_entity": [
    [
      0.59375,
      0.3114754098360656,
      0.40860215053763443
    ],
    [
      0.8152524167561761,
      0.7047353760445683,
      0.7559760956175299
    ],
    [
      0.8589341692789969,
      0.6674786845310596,
      0.7511994516792323
    ],
    [
      0.6359447004608295,
      0.4423076923076923,
      0.5217391304347826
    ]
  ],
  "p_r_f1_all_entity_ELB": [
    0.8016216216216216,
    0.635934819897084,
    0.7092300334768054
  ],
  "p_r_f1_link_ELK": [
    0.6720977596741344,
    0.3101503759398496,
    0.42443729903536975
  ]
}

Prediction output

{
    "data_id": "00081",
    "text_unit": ["1", "SU", "##RI", "##MI","29", ... ],
    "pr_parse": [
      [{"menu.nm": "SURIMI"}, {"menu.cnt": "1"}, {"menu.price": "29,091"}], 
      [{"menu.nm": "CREAMY CHK CLS FTC"}, {"menu.cnt": "1"}, {"menu.price": "42,727"}],
      [{"menu.nm": "MIX 4FUN CHOCOLATE"}, {"menu.cnt": "1"}], 
      [{"menu.nm": "GREEN ITSODA PITCHER"}, {"menu.price": "19,091"}, {"menu.cnt": "1"}], 
      [{"menu.nm": "SC/R GRILLED STEAK"}, {"menu.cnt": "1"}, {"menu.price": "99,091"}], 
      [{"sub_total.subtotal_price": "250,909"}, {"sub_total.tax_price": "25,091"}], 
      [{"total.total_price": "276,000"}]],
    "pr_label": [
      [
        [
          1,
          0,
          0,
          ...
         ],
         ... 
        ]
      ]
    "pr_text_unit_field_label": ["menu.cnt","menu.nm","menu.nm","menu.nm", "menu.price",...]
}

Preprocessing

CORD

FUNSD

Model

Training

Evaluation

bash scripts/test_[task].sh

Prediction

bash scripts/predict_cord.sh

Citation

@inproceedings{hwang2021spade,
    title = "Spatial Dependency Parsing for Semi-Structured Document Information Extraction",
      author = {Wonseok Hwang and
               Jinyeung Yim and
               Seunghyun Park and
               Sohee Yang and
               Minjoon Seo},
    booktitle = "ACL",
    year = {2021}
}

License

Copyright 2021-present NAVER Corp.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an " AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.