search

DoodleJZ / HPSG-Neural-Parser

Source code for "Head-Driven Phrase Structure Grammar Parsing on Penn Treebank" published at ACL 2019
https://arxiv.org/abs/1907.02684
MIT License
107 stars 25 forks source link

Test set accuracy is very low #3

Open nangeblog opened 5 years ago

nangeblog commented 5 years ago

Loading model from models/joint_bert_dev=95.55_devuas=96.67_devlas=94.86.pt... Reading dependency parsing data from data/ptb_test_3.3.0.sd Loading test trees from data/23.auto.clean... Loaded 2,416 test examples. Parsing test sentences... test-fscore (Recall=6.51, Precision=6.70, FScore=6.61) test-elapsed 0h00m34s best test W. Punct: ucorr: 7738, lcorr: 1426, total: 56684, uas: 13.65%, las: 2.52%, ucm: 1.74%, lcm: 0.75% best test Wo Punct: ucorr: 7150, lcorr: 1256, total: 49893, uas: 14.33%, las: 2.52%, ucm: 1.82%, lcm: 0.75% best test Root: corr: 2416, total: 2416, acc: 100.00%

DoodleJZ commented 5 years ago

Thank for showing the issues. Since we update with pytorch_transformers package, it has some problems. Now we use the previous vision of pretrained_bert and everything is ok. Later, We will find the problems and update the code.