jiesutd / LatticeLSTM

Chinese NER using Lattice LSTM. Code for ACL 2018 paper.
1.8k stars 453 forks source link

hello,After running your code, the f value is only 0.4,sorry,Is there anything I should pay attention to when running #44

Closed Biubiulity closed 5 years ago

Biubiulity commented 5 years ago

please tell me why?

jiesutd commented 5 years ago

Please give more details, then can I give you advice. And please do not repeatly close-reopen issue, it will cause spam in email system.

Biubiulity commented 5 years ago

Please give more details, then can I give you advice. And please do not repeatly close-reopen issue, it will cause spam in email system.

I'm really sorry.Because this is the first time I used Github's comment,so I don't know use it others can see it.Please forgive me. And I want to know:

  1. I want to know whether the data set in data in your code is the data set of weibo. 2.Does your code run other models as well, for example word_baseline ?If yes ,Could you tell me how to modify it? 3.when I run the dataset in your code ,the F_score is 0.4177,so I want to know whether the data set I am running is the data set of weibo. The log file is: D:\Anaconda3\python.exe G:/experiment/main.py CuDNN: True GPU available: False Status: train Seg: True Train file: ./data/demo.train.char Dev file: ./data/demo.dev.char Test file: ./data/demo.test.char Raw file: ../data/onto4ner.cn/demo.test.char Char emb: data/gigaword_chn.all.a2b.uni.ite50.vec Bichar emb: None Gaz file: data/ctb.50d.vec Model saved to: data/demo Load gaz file: data/ctb.50d.vec total size: 704368 gaz alphabet size: 10130 gaz alphabet size: 10966 gaz alphabet size: 13652 build word pretrain emb... Embedding: pretrain word:11327, prefect match:2551, case_match:0, oov:26, oov%:0.01008533747090768 build biword pretrain emb... Embedding: pretrain word:0, prefect match:0, case_match:0, oov:31748, oov%:0.9999685029449746 build gaz pretrain emb... Embedding: pretrain word:704368, prefect match:13650, case_match:0, oov:1, oov%:7.32493407559332e-05 Training model... DATA SUMMARY START: Tag scheme: BMES MAX SENTENCE LENGTH: 250 MAX WORD LENGTH: -1 Number normalized: True Use bigram: False Word alphabet size: 2578 Biword alphabet size: 31749 Char alphabet size: 2578 Gaz alphabet size: 13652 Label alphabet size: 18 Word embedding size: 50 Biword embedding size: 50 Char embedding size: 30 Gaz embedding size: 50 Norm word emb: True Norm biword emb: True Norm gaz emb: False Norm gaz dropout: 0.5 Train instance number: 1147 Dev instance number: 113 Test instance number: 316 Raw instance number: 0 Hyperpara iteration: 100 Hyperpara batch size: 1 Hyperpara lr: 0.015 Hyperpara lr_decay: 0.05 Hyperpara HP_clip: 5.0 Hyperpara momentum: 0 Hyperpara hidden_dim: 200 Hyperpara dropout: 0.5 Hyperpara lstm_layer: 1 Hyperpara bilstm: True Hyperpara GPU: False Hyperpara use_gaz: True Hyperpara fix gaz emb: False Hyperpara use_char: False DATA SUMMARY END. Data setting saved to file: data/demo.dset build batched lstmcrf... build batched bilstm... build LatticeLSTM... forward , Fix emb: False gaz drop: 0.5 load pretrain word emb... (13652, 50) build LatticeLSTM... backward , Fix emb: False gaz drop: 0.5 load pretrain word emb... (13652, 50) build batched crf... finished built model. Epoch: 0/100 Learning rate is setted as: 0.015 Instance: 500; Time: 140.08s; loss: 9362.1726; acc: 18641/21743=0.8573 Instance: 1000; Time: 123.59s; loss: 5484.8606; acc: 37797/43593=0.8670 Instance: 1147; Time: 33.99s; loss: 1021.0708; acc: 43146/49471=0.8721 Epoch: 0 training finished. Time: 297.66s, speed: 3.85st/s, total loss: 15868.103970527649 gold_num = 215 pred_num = 102 right_num = 69 Dev: time: 3.70s, speed: 30.60st/s; acc: 0.9162, p: 0.6765, r: 0.3209, f: 0.4353 Exceed previous best f score: -1 gold_num = 340 pred_num = 109 right_num = 55 Test: time: 9.94s, speed: 31.95st/s; acc: 0.9317, p: 0.5046, r: 0.1618, f: 0.2450 Epoch: 1/100 Learning rate is setted as: 0.014249999999999999 Instance: 500; Time: 121.06s; loss: 3302.7600; acc: 19684/21682=0.9078 Instance: 1000; Time: 113.08s; loss: 3065.0727; acc: 39064/42920=0.9102 Instance: 1147; Time: 35.27s; loss: 912.5710; acc: 45036/49471=0.9104 Epoch: 1 training finished. Time: 269.42s, speed: 4.26st/s, total loss: 7280.403723716736 gold_num = 215 pred_num = 149 right_num = 105 Dev: time: 3.43s, speed: 33.00st/s; acc: 0.9267, p: 0.7047, r: 0.4884, f: 0.5769 Exceed previous best f score: 0.43533123028391163 gold_num = 340 pred_num = 186 right_num = 103 Test: time: 8.56s, speed: 37.14st/s; acc: 0.9410, p: 0.5538, r: 0.3029, f: 0.3916 ... ... Epoch: 99/100 Learning rate is setted as: 9.348204032106312e-05 Instance: 500; Time: 115.19s; loss: 281.3181; acc: 21674/21864=0.9913 Instance: 1000; Time: 109.61s; loss: 273.0064; acc: 42363/42772=0.9904 Instance: 1147; Time: 34.63s; loss: 64.7753; acc: 49009/49471=0.9907 Epoch: 99 training finished. Time: 259.43s, speed: 4.42st/s, total loss: 619.0996932983398 gold_num = 215 pred_num = 144 right_num = 117 Dev: time: 3.36s, speed: 33.64st/s; acc: 0.9382, p: 0.8125, r: 0.5442, f: 0.6518 gold_num = 340 pred_num = 201 right_num = 113 Test: time: 8.18s, speed: 38.68st/s; acc: 0.9433, p: 0.5622, r: 0.3324, f: 0.4177

Process finished with exit code 0 Since this is my first time to touch pytorch, read codes is difficult to me .So I don't know what'wrong with them . If you have time, please find time to answer my questions.Thank you very much !

jiesutd commented 5 years ago

Train file: ./data/demo.train.char Dev file: ./data/demo.dev.char Test file: ./data/demo.test.char

You used the wrong data, this is the demo data which is to test if you can run the code. Please us the correct data as the input.

Biubiulity commented 5 years ago

OK, I know ,Thank you for your patient answer