some wrongs when I use bert

[SaintLogos1234]

I run Bert the way you do, But the following bug has appeared:

2019-11-15 10:35:32 - INFO - DEETask - ====================Check Setting Validity==================== 2019-11-15 10:35:32 - INFO - DEETask - Setting: { "data_dir": "./Data", "model_dir": "./Exps/train_debug/Model", "output_dir": "./Exps/train_debug/Output", "bert_model": "/home/luandong/xieyangyang/model/bert_model/bert-chinese-base", "train_file_name": "train.json", "dev_file_name": "dev.json", "test_file_name": "test.json", "max_seq_len": 128, "train_batch_size": 8, "eval_batch_size": 2, "learning_rate": 0.0001, "num_train_epochs": 100, "warmup_proportion": 0.1, "no_cuda": false, "local_rank": -1, "seed": 99, "gradient_accumulation_steps": 16, "optimize_on_cpu": false, "fp16": false, "loss_scale": 128, "cpt_file_name": "Doc2EDAG", "summary_dir_name": "/tmp/Summary", "max_sent_len": 128, "max_sent_num": 64, "use_bert": true, "only_master_logging": true, "resume_latest_cpt": true, "model_type": "Doc2EDAG", "rearrange_sent": false, "use_crf_layer": true, "min_teacher_prob": 0.1, "schedule_epoch_start": 10, "schedule_epoch_length": 10, "loss_lambda": 0.05, "loss_gamma": 1.0, "add_greedy_dec": true, "use_token_role": true, "seq_reduce_type": "MaxPooling", "hidden_size": 768, "dropout": 0.1, "ff_size": 1024, "num_tf_layers": 4, "use_path_mem": true, "use_scheduled_sampling": true, "use_doc_enc": true, "neg_field_loss_scaling": 3.0 } 2019-11-15 10:35:32 - INFO - DEETask - ====================Init Device==================== 2019-11-15 10:35:32 - INFO - DEETask - device cuda n_gpu 1 distributed training False 2019-11-15 10:35:32 - INFO - DEETask - ====================Reset Random Seed to 99==================== 2019-11-15 10:35:32 - INFO - DEETask - Initializing DEETask 2019-11-15 10:35:32 - INFO - pytorch_pretrained_bert.tokenization - loading vocabulary file /home/luandong/xieyangyang/model/bert_model/bert-chinese-base/vocab.txt 2019-11-15 10:35:32 - INFO - DEETask - ====================Load Task Data==================== 2019-11-15 10:35:32 - INFO - DEETask - Load train portion 2019-11-15 10:35:32 - INFO - DEETask - Load example feature dataset from ./Data/train.json 2019-11-15 10:36:45 - INFO - dee.dee_helper - 25632 documents, ignore 0 examples, truncate 0 docs, 29342 sents, 21289 spans 2019-11-15 10:36:45 - INFO - DEETask - Load dev portion 2019-11-15 10:36:45 - INFO - DEETask - Load example feature dataset from ./Data/dev.json 2019-11-15 10:36:56 - INFO - dee.dee_helper - 3204 documents, ignore 0 examples, truncate 0 docs, 4583 sents, 3049 spans 2019-11-15 10:36:56 - INFO - DEETask - Load test portion 2019-11-15 10:36:56 - INFO - DEETask - Load example feature dataset from ./Data/test.json 2019-11-15 10:37:07 - INFO - dee.dee_helper - 3204 documents, ignore 0 examples, truncate 0 docs, 4331 sents, 2527 spans 2019-11-15 10:37:07 - INFO - pytorch_pretrained_bert.modeling - loading archive file /home/luandong/xieyangyang/model/bert_model/bert-chinese-base 2019-11-15 10:37:07 - INFO - pytorch_pretrained_bert.modeling - Model config { "attention_probs_dropout_prob": 0.1, "directionality": "bidi", "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "initializer_range": 0.02, "intermediate_size": 3072, "max_position_embeddings": 512, "num_attention_heads": 12, "num_hidden_layers": 12, "pooler_fc_size": 768, "pooler_num_attention_heads": 12, "pooler_num_fc_layers": 3, "pooler_size_per_head": 128, "pooler_type": "first_token_transform", "type_vocab_size": 2, "vocab_size": 21128 }

Traceback (most recent call last): File "run_dee_task.py", line 62, in dee_task = DEETask(dee_setting, load_train=not in_argv.skip_train) File "/home/luandong/xieyangyang/reaserch/nlp_reaserch/Doc2EDAG-master/dee/dee_task.py", line 121, in init self.setting.bert_model, self.setting.num_entity_labels File "/home/luandong/anaconda3/envs/xyy_pytorch_1.1.0/lib/python3.6/site-packages/pytorch_pretrained_bert/modeling.py", line 506, in from_pretrained model = cls(config, *inputs, **kwargs) TypeError: init() missing 1 required positional argument: 'num_entity_labels'

my torch version is :1.1.0, GPU: RTX2080Ti

[shun-zheng]

Can you run the program without using BERT?

BTW, we utilize pytorch_pretrained_bert == 0.4.0 to use BERT (maybe a little old, but welcome pull request to add support for new BERT versions), so please ensure you have installed the correct version.

[SaintLogos1234]

I run the program without using BERT， it works well, but I run the program with BERT, the above bug appears, my pytorch-pretrained-bert version is 0.4.0

[shun-zheng]

Thanks for reporting this bug about supporting BERT.

Since we just test the basic BERT feature many months ago, the BERT support of this version may be too old. So please use the basic program without BERT first, I may refactor the BERT part later (maybe in the next month).

[shun-zheng]

Thanks for your PR. Sorry for the late merge.