VinAIResearch / PhoNLP

PhoNLP: A BERT-based multi-task learning model for part-of-speech tagging, named entity recognition and dependency parsing (NAACL 2021)
Apache License 2.0
135 stars 18 forks source link

PhoNLP doesn't train or evaluate the NER Task #9

Closed COCOMiss closed 1 year ago

COCOMiss commented 1 year ago

I am running the code of PhoNLP. However , the result shows it doesn't train or evaluate NER task. I paste my parameters below: Train: python3 phonlp/models/run_phonlp.py --mode train --save_dir ./phonlp_tmp_save_model \ --pretrained_lm "vinai/phobert-base" \ --lr 1e-5 --batch_size 32 --num_epoch 40 \ --lambda_pos 0.4 --lambda_ner 0.2 --lambda_dep 0.4 \ --train_file_pos phonlp/sample_data/pos_train.txt --eval_file_pos phonlp/sample_data/pos_valid.txt \ --train_file_ner phonlp/sample_data/ner_train.txt --eval_file_ner phonlp/sample_data/ner_valid.txt \ --train_file_dep phonlp/sample_data/dep_train.conll --eval_file_dep phonlp/sample_data/dep_valid.conll\ --output_file_dep phonlp/models/jointmodel/dep.out

Evaluate: python3 phonlp/models/run_phonlp.py --mode eval --save_dir ./phonlp_tmp_save_model \ --batch_size 8 \ --eval_file_pos phonlp/sample_data/pos_test.txt \ --eval_file_ner phonlp/sample_data/ner_test.txt \ --eval_file_dep phonlp/sample_data/dep_test.conll\ --output_file_dep phonlp/models/jointmodel/dep.out

The results of train and evaluate: Train Result: Training ended with 39 epochs. Best dev las = 5.23, uas = 15.28, upos = 24.22, f1 = 0.0

Evaluate Result: POS tagging: 24.25, NER: 0.00, Dependency parsing: 6.63/27.24

I would appreciate it for your reply. @datquocnguyen @thelinhbkhn2014 @dangne @maihoangdao Thank you~~

datquocnguyen commented 1 year ago

Hi, the "sample" data consists of a few sentences for command line illustration only. Thus, it is not surprising that you'd obtain NER f1 at 0.0. You should replace the "sample" data with your training data.

COCOMiss commented 1 year ago

Hi~~ I have set my train mode for NER --train_file_ner phonlp/sample_data/ner_train.txt --eval_file_ner phonlp/sample_data/ner_valid.txt However, it doesn't work And I tried to set the test mode for NER --eval_file_ner phonlp/sample_data/ner_train.txt or --eval_file_ner phonlp/sample_data/ner_test.txt

All results show NER 0.0

Should I change another dataset or do something else? Thank you for your patience~~

@datquocnguyen