bugface
commented
1 year ago we need to revisit XLNet, the implementation has problems where the performances are very low consistently cross different datasets. So the new features are not integrated with XLNet.
python src/run_transformer_ner.py \ --model_type xlnet \ --pretrained_model xlnet-base-cased \ --data_dir ./test_data/conll-2003 \ --new_model_dir ./new_bert_ner_model \ --overwrite_model_dir \ --predict_output_file ./bert_pred.txt \ --max_seq_length 256 \ --save_model_core \ --do_train \ --do_predict \ --model_selection_scoring strict-f_score-1 \ --do_lower_case \ --train_batch_size 8 \ --eval_batch_size 8 \ --train_steps 500 \ --learning_rate 1e-5 \ --num_train_epochs 1 \ --gradient_accumulation_steps 1 \ --do_warmup \ --seed 13 \ --warmup_ratio 0.1 \ --max_num_checkpoints 3 \ --log_file ./log.txt \ --progress_bar \ --early_stop 3
Traceback (most recent call last): File "/data/datasets/yonghui/project/ClinicalTransformerNER/src/run_transformer_ner.py", line 169, in main run_task(global_args) File "/data/datasets/yonghui/project/ClinicalTransformerNER/src/transformer_ner/task.py", line 604, in run_task model = model_model.from_pretrained(args.pretrained_model, config=config) File "/home/yonghui.wu/.pyenv/versions/anaconda3-2021.11/lib/python3.9/site-packages/transformers/modeling_utils.py", line 2024, in from_pretrained model = cls(config, *model_args, **model_kwargs) File "/data/datasets/yonghui/project/ClinicalTransformerNER/src/transformer_ner/model.py", line 308, in init if config.use_biaffine: File "/home/yonghui.wu/.pyenv/versions/anaconda3-2021.11/lib/python3.9/site-packages/transformers/configuration_utils.py", line 253, in getattribute return super().getattribute(key) AttributeError: 'XLNetConfig' object has no attribute 'use_biaffine'