Any instruction for running 'run_tagger_plus.py'?

[luciusssss]

Hi! I run the 'run_tagger_plus.py' file with the same command for 'run_tagger.py', as provided in the README.

python run_tagger_plus.py \
    --model_name_or_path ../models/bert-base-uncased \
    --data_dir ../data/MultiSpanQA_data \
    --output_dir ../output/tagger-plus_bert-base \
    --overwrite_output_dir \
    --overwrite_cache \
    --do_train \
    --do_eval \
    --per_device_train_batch_size 4 \
    --eval_accumulation_steps 50 \
    --learning_rate 3e-5 \
    --num_train_epochs 3 \
    --max_seq_length  512 \
    --doc_stride 128

However, the model performance on the valid set seems low.

{'em_precision': 24.838709677419356, 'em_recall': 16.117216117216117, 'em_f1': 19.549349412884798, 'overlap_precision': 50.70855308353936, 'overlap_recall': 24.571631786577246, 'overlap_f1': 33.10278520034419}

Maybe there are some mistakes in the way I run this code. Could you please provide some instructions for running the tagger-plus model?

Thanks!

[haonan-li]

I think that is because the default lambda for loss is too large, change span_lambda to 1 you will get a normal result.

[luciusssss]

Thanks a lot! It works.