search

asappresearch / slue-toolkit

A toolkit for Spoken Language Understanding Evaluation (SLUE) benchmark. Refer paper https://arxiv.org/abs/2111.10367 for more details. Official website: https://asappresearch.github.io/slue-toolkit/
https://asappresearch.github.io/slue-toolkit/
MIT License
62 stars 16 forks source link

Fixes to text NER and pipeline scripts #17

Closed ankitapasad closed 2 years ago

ankitapasad commented 2 years ago

  1. Resolved issue #16 (i) Moved "handling the apostrophe" snippet to slue-toolkit/text_ner/reformat_pipeline.py.

  2. The tag-id mapping files are now being produced in slue_toolkit/prepare/prepare_voxpopuli.py (instead of ner_deberta_modules.py).

  3. Cleaned up and tested the text NER training and evaluation codes are now bug-free.

  4. Cleaned up pipeline scripts

  5. Reformated with black

Note that you'd need to run python slue_toolkit/prepare/prepare_voxpopuli.py create_manifest to update the processed data. This will add the tag-id mapping files to the data/ directory.