(Unofficial) Pytorch implementation of JointBERT
: BERT for Joint Intent Classification and Slot Filling
intent
and slot
at the same time from one BERT model (=Joint model)--slot_loss_coef
option)--use_crf
optionTrain | Dev | Test | Intent Labels | Slot Labels | |
---|---|---|---|---|---|
ATIS | 4,478 | 500 | 893 | 21 | 120 |
Snips | 13,084 | 700 | 700 | 7 | 72 |
UNK
for labels (For intent and slot labels which are only shown in dev and test dataset)PAD
for slot label$ python3 main.py --task {task_name} \
--model_type {model_type} \
--model_dir {model_dir_name} \
--do_train --do_eval \
--use_crf
# For ATIS
$ python3 main.py --task atis \
--model_type bert \
--model_dir atis_model \
--do_train --do_eval
# For Snips
$ python3 main.py --task snips \
--model_type bert \
--model_dir snips_model \
--do_train --do_eval
$ python3 predict.py --input_file {INPUT_FILE_PATH} --output_file {OUTPUT_FILE_PATH} --model_dir {SAVED_CKPT_PATH}
uncased
modelIntent acc (%) | Slot F1 (%) | Sentence acc (%) | ||
---|---|---|---|---|
Snips | BERT | 99.14 | 96.90 | 93.00 |
BERT + CRF | 98.57 | 97.24 | 93.57 | |
DistilBERT | 98.00 | 96.10 | 91.00 | |
DistilBERT + CRF | 98.57 | 96.46 | 91.85 | |
ALBERT | 98.43 | 97.16 | 93.29 | |
ALBERT + CRF | 99.00 | 96.55 | 92.57 | |
ATIS | BERT | 97.87 | 95.59 | 88.24 |
BERT + CRF | 97.98 | 95.93 | 88.58 | |
DistilBERT | 97.76 | 95.50 | 87.68 | |
DistilBERT + CRF | 97.65 | 95.89 | 88.24 | |
ALBERT | 97.64 | 95.78 | 88.13 | |
ALBERT + CRF | 97.42 | 96.32 | 88.69 |
sentence-level semantic frame accuracy