search

macanv / BERT-BiLSTM-CRF-NER

Tensorflow solution of NER task Using BiLSTM-CRF model with Google BERT Fine-tuning And private Server services
https://github.com/macanv/BERT-BiLSMT-CRF-NER
4.72k stars 1.26k forks source link

在自己的数据和标签集上进行了一下测试，如何提高效果？ #147

Open Vincen1994 opened 5 years ago

Vincen1994 commented 5 years ago

想利用这个模型做序列标注的分词、POS和NER联合任务，然后在自己的14000句数据上跑了一下训练，10000句训练、2000dev和2000test，结果如下：

processed 35069 tokens with 17629 phrases; found: 19837 phrases; correct: 13586. 71 accuracy: 81.61%; precision: 68.49%; recall: 77.07%; FB1: 72.52 72 LOC: precision: 60.37%; recall: 72.93%; FB1: 66.06 598 73 ORG: precision: 57.86%; recall: 78.28%; FB1: 66.54 299 74 PER: precision: 28.04%; recall: 42.25%; FB1: 33.71 642 75 TIME: precision: 70.75%; recall: 75.25%; FB1: 72.93 318 76 a: precision: 38.10%; recall: 46.47%; FB1: 41.87 622 77 ad: precision: 39.25%; recall: 56.00%; FB1: 46.15 107 78 an: precision: 0.00%; recall: 0.00%; FB1: 0.00 21 79 c: precision: 84.44%; recall: 88.03%; FB1: 86.20 270 80 d: precision: 60.43%; recall: 60.73%; FB1: 60.58 604 81 f: precision: 60.40%; recall: 58.10%; FB1: 59.22 202 82 m: precision: 80.55%; recall: 87.07%; FB1: 83.68 694 83 n: precision: 69.52%; recall: 75.72%; FB1: 72.49 3862 84 nr: precision: 0.34%; recall: 1.49%; FB1: 0.55 298 85 ns: precision: 8.45%; recall: 17.91%; FB1: 11.48 142 86 nt: precision: 0.00%; recall: 0.00%; FB1: 0.00 25 87 nw: precision: 0.00%; recall: 0.00%; FB1: 0.00 173 88 nz: precision: 17.43%; recall: 39.71%; FB1: 24.22 1119 89 p: precision: 91.72%; recall: 91.57%; FB1: 91.65 604 90 q: precision: 0.00%; recall: 0.00%; FB1: 0.00 0 91 r: precision: 85.52%; recall: 90.83%; FB1: 88.10 822 92 s: precision: 59.00%; recall: 67.05%; FB1: 62.77 100 93 t: precision: 41.51%; recall: 45.36%; FB1: 43.35 106 94 u: precision: 95.88%; recall: 96.36%; FB1: 96.12 800 95 v: precision: 76.32%; recall: 81.37%; FB1: 78.76 3387 96 vd: precision: 0.00%; recall: 0.00%; FB1: 0.00 36 97 vn: precision: 46.43%; recall: 49.76%; FB1: 48.04 672 98 w: precision: 96.44%; recall: 95.69%; FB1: 96.07 3177 99 xc: precision: 82.48%; recall: 76.35%; FB1: 79.30 137

效果上可能是由于标签太多？另外数据集规模可能也不够大，从参数上这种情况怎么调整可能能取得更好的效果呢？谢谢！

xmy7216 commented 5 years ago

想利用这个模型做序列标注的分词、POS和NER联合任务，然后在自己的14000句数据上跑了一下训练，10000句训练、2000dev和2000test，结果如下：

processed 35069 tokens with 17629 phrases; found: 19837 phrases; correct: 13586. 71 accuracy: 81.61%; precision: 68.49%; recall: 77.07%; FB1: 72.52 72 LOC: precision: 60.37%; recall: 72.93%; FB1: 66.06 598 73 ORG: precision: 57.86%; recall: 78.28%; FB1: 66.54 299 74 PER: precision: 28.04%; recall: 42.25%; FB1: 33.71 642 75 TIME: precision: 70.75%; recall: 75.25%; FB1: 72.93 318 76 a: precision: 38.10%; recall: 46.47%; FB1: 41.87 622 77 ad: precision: 39.25%; recall: 56.00%; FB1: 46.15 107 78 an: precision: 0.00%; recall: 0.00%; FB1: 0.00 21 79 c: precision: 84.44%; recall: 88.03%; FB1: 86.20 270 80 d: precision: 60.43%; recall: 60.73%; FB1: 60.58 604 81 f: precision: 60.40%; recall: 58.10%; FB1: 59.22 202 82 m: precision: 80.55%; recall: 87.07%; FB1: 83.68 694 83 n: precision: 69.52%; recall: 75.72%; FB1: 72.49 3862 84 nr: precision: 0.34%; recall: 1.49%; FB1: 0.55 298 85 ns: precision: 8.45%; recall: 17.91%; FB1: 11.48 142 86 nt: precision: 0.00%; recall: 0.00%; FB1: 0.00 25 87 nw: precision: 0.00%; recall: 0.00%; FB1: 0.00 173 88 nz: precision: 17.43%; recall: 39.71%; FB1: 24.22 1119 89 p: precision: 91.72%; recall: 91.57%; FB1: 91.65 604 90 q: precision: 0.00%; recall: 0.00%; FB1: 0.00 0 91 r: precision: 85.52%; recall: 90.83%; FB1: 88.10 822 92 s: precision: 59.00%; recall: 67.05%; FB1: 62.77 100 93 t: precision: 41.51%; recall: 45.36%; FB1: 43.35 106 94 u: precision: 95.88%; recall: 96.36%; FB1: 96.12 800 95 v: precision: 76.32%; recall: 81.37%; FB1: 78.76 3387 96 vd: precision: 0.00%; recall: 0.00%; FB1: 0.00 36 97 vn: precision: 46.43%; recall: 49.76%; FB1: 48.04 672 98 w: precision: 96.44%; recall: 95.69%; FB1: 96.07 3177 99 xc: precision: 82.48%; recall: 76.35%; FB1: 79.30 137

效果上可能是由于标签太多？另外数据集规模可能也不够大，从参数上这种情况怎么调整可能能取得更好的效果呢？谢谢！

您好，我也在自己的数据集和标签上进行实验，得到的结果全是0。对于数据集，模型输入是每行一个字以及对应的标签，以空格隔开，句子与句子之间用空行隔开，除了这些处理，请问您在自己的数据集上还做了哪些处理呢？

lln1997 commented 4 years ago

想利用这个模型做序列标注的分词、POS和NER联合任务，然后在自己的14000句数据上跑了一下训练，10000句训练、2000dev和2000test，结果如下： processed 35069 tokens with 17629 phrases; found: 19837 phrases; correct: 13586. 71 accuracy: 81.61%; precision: 68.49%; recall: 77.07%; FB1: 72.52 72 LOC: precision: 60.37%; recall: 72.93%; FB1: 66.06 598 73 ORG: precision: 57.86%; recall: 78.28%; FB1: 66.54 299 74 PER: precision: 28.04%; recall: 42.25%; FB1: 33.71 642 75 TIME: precision: 70.75%; recall: 75.25%; FB1: 72.93 318 76 a: precision: 38.10%; recall: 46.47%; FB1: 41.87 622 77 ad: precision: 39.25%; recall: 56.00%; FB1: 46.15 107 78 an: precision: 0.00%; recall: 0.00%; FB1: 0.00 21 79 c: precision: 84.44%; recall: 88.03%; FB1: 86.20 270 80 d: precision: 60.43%; recall: 60.73%; FB1: 60.58 604 81 f: precision: 60.40%; recall: 58.10%; FB1: 59.22 202 82 m: precision: 80.55%; recall: 87.07%; FB1: 83.68 694 83 n: precision: 69.52%; recall: 75.72%; FB1: 72.49 3862 84 nr: precision: 0.34%; recall: 1.49%; FB1: 0.55 298 85 ns: precision: 8.45%; recall: 17.91%; FB1: 11.48 142 86 nt: precision: 0.00%; recall: 0.00%; FB1: 0.00 25 87 nw: precision: 0.00%; recall: 0.00%; FB1: 0.00 173 88 nz: precision: 17.43%; recall: 39.71%; FB1: 24.22 1119 89 p: precision: 91.72%; recall: 91.57%; FB1: 91.65 604 90 q: precision: 0.00%; recall: 0.00%; FB1: 0.00 0 91 r: precision: 85.52%; recall: 90.83%; FB1: 88.10 822 92 s: precision: 59.00%; recall: 67.05%; FB1: 62.77 100 93 t: precision: 41.51%; recall: 45.36%; FB1: 43.35 106 94 u: precision: 95.88%; recall: 96.36%; FB1: 96.12 800 95 v: precision: 76.32%; recall: 81.37%; FB1: 78.76 3387 96 vd: precision: 0.00%; recall: 0.00%; FB1: 0.00 36 97 vn: precision: 46.43%; recall: 49.76%; FB1: 48.04 672 98 w: precision: 96.44%; recall: 95.69%; FB1: 96.07 3177 99 xc: precision: 82.48%; recall: 76.35%; FB1: 79.30 137 效果上可能是由于标签太多？另外数据集规模可能也不够大，从参数上这种情况怎么调整可能能取得更好的效果呢？谢谢！

您好，我也在自己的数据集和标签上进行实验，得到的结果全是0。对于数据集，模型输入是每行一个字以及对应的标签，以空格隔开，句子与句子之间用空行隔开，除了这些处理，请问您在自己的数据集上还做了哪些处理呢？

您好，请问您解决了全是0的结果了嘛

qzd147 commented 4 years ago

想利用这个模型做序列标注的分词、POS和NER联合任务，然后在自己的14000句数据上跑了一下训练，10000句训练、2000dev和2000test，结果如下： processed 35069 tokens with 17629 phrases; found: 19837 phrases; correct: 13586. 71 accuracy: 81.61%; precision: 68.49%; recall: 77.07%; FB1: 72.52 72 LOC: precision: 60.37%; recall: 72.93%; FB1: 66.06 598 73 ORG: precision: 57.86%; recall: 78.28%; FB1: 66.54 299 74 PER: precision: 28.04%; recall: 42.25%; FB1: 33.71 642 75 TIME: precision: 70.75%; recall: 75.25%; FB1: 72.93 318 76 a: precision: 38.10%; recall: 46.47%; FB1: 41.87 622 77 ad: precision: 39.25%; recall: 56.00%; FB1: 46.15 107 78 an: precision: 0.00%; recall: 0.00%; FB1: 0.00 21 79 c: precision: 84.44%; recall: 88.03%; FB1: 86.20 270 80 d: precision: 60.43%; recall: 60.73%; FB1: 60.58 604 81 f: precision: 60.40%; recall: 58.10%; FB1: 59.22 202 82 m: precision: 80.55%; recall: 87.07%; FB1: 83.68 694 83 n: precision: 69.52%; recall: 75.72%; FB1: 72.49 3862 84 nr: precision: 0.34%; recall: 1.49%; FB1: 0.55 298 85 ns: precision: 8.45%; recall: 17.91%; FB1: 11.48 142 86 nt: precision: 0.00%; recall: 0.00%; FB1: 0.00 25 87 nw: precision: 0.00%; recall: 0.00%; FB1: 0.00 173 88 nz: precision: 17.43%; recall: 39.71%; FB1: 24.22 1119 89 p: precision: 91.72%; recall: 91.57%; FB1: 91.65 604 90 q: precision: 0.00%; recall: 0.00%; FB1: 0.00 0 91 r: precision: 85.52%; recall: 90.83%; FB1: 88.10 822 92 s: precision: 59.00%; recall: 67.05%; FB1: 62.77 100 93 t: precision: 41.51%; recall: 45.36%; FB1: 43.35 106 94 u: precision: 95.88%; recall: 96.36%; FB1: 96.12 800 95 v: precision: 76.32%; recall: 81.37%; FB1: 78.76 3387 96 vd: precision: 0.00%; recall: 0.00%; FB1: 0.00 36 97 vn: precision: 46.43%; recall: 49.76%; FB1: 48.04 672 98 w: precision: 96.44%; recall: 95.69%; FB1: 96.07 3177 99 xc: precision: 82.48%; recall: 76.35%; FB1: 79.30 137 效果上可能是由于标签太多？另外数据集规模可能也不够大，从参数上这种情况怎么调整可能能取得更好的效果呢？谢谢！

您好，我也在自己的数据集和标签上进行实验，得到的结果全是0。对于数据集，模型输入是每行一个字以及对应的标签，以空格隔开，句子与句子之间用空行隔开，除了这些处理，请问您在自己的数据集上还做了哪些处理呢？

您好，请问您解决了全是0的结果了嘛

同问，全是0怎么解决的？我也是用自己的数据集全是零，目前不知所措。

malijun1220 commented 2 years ago

想利用这个模型做序列标注的分词、POS和NER联合任务，然后在自己的14000句数据上跑了一下训练，10000句训练、2000dev和2000test，结果如下： processed 35069 tokens with 17629 phrases; found: 19837 phrases; correct: 13586. 71 accuracy: 81.61%; precision: 68.49%; recall: 77.07%; FB1: 72.52 72 LOC: precision: 60.37%; recall: 72.93%; FB1: 66.06 598 73 ORG: precision: 57.86%; recall: 78.28%; FB1: 66.54 299 74 PER: precision: 28.04%; recall: 42.25%; FB1: 33.71 642 75 TIME: precision: 70.75%; recall: 75.25%; FB1: 72.93 318 76 a: precision: 38.10%; recall: 46.47%; FB1: 41.87 622 77 ad: precision: 39.25%; recall: 56.00%; FB1: 46.15 107 78 an: precision: 0.00%; recall: 0.00%; FB1: 0.00 21 79 c: precision: 84.44%; recall: 88.03%; FB1: 86.20 270 80 d: precision: 60.43%; recall: 60.73%; FB1: 60.58 604 81 f: precision: 60.40%; recall: 58.10%; FB1: 59.22 202 82 m: precision: 80.55%; recall: 87.07%; FB1: 83.68 694 83 n: precision: 69.52%; recall: 75.72%; FB1: 72.49 3862 84 nr: precision: 0.34%; recall: 1.49%; FB1: 0.55 298 85 ns: precision: 8.45%; recall: 17.91%; FB1: 11.48 142 86 nt: precision: 0.00%; recall: 0.00%; FB1: 0.00 25 87 nw: precision: 0.00%; recall: 0.00%; FB1: 0.00 173 88 nz: precision: 17.43%; recall: 39.71%; FB1: 24.22 1119 89 p: precision: 91.72%; recall: 91.57%; FB1: 91.65 604 90 q: precision: 0.00%; recall: 0.00%; FB1: 0.00 0 91 r: precision: 85.52%; recall: 90.83%; FB1: 88.10 822 92 s: precision: 59.00%; recall: 67.05%; FB1: 62.77 100 93 t: precision: 41.51%; recall: 45.36%; FB1: 43.35 106 94 u: precision: 95.88%; recall: 96.36%; FB1: 96.12 800 95 v: precision: 76.32%; recall: 81.37%; FB1: 78.76 3387 96 vd: precision: 0.00%; recall: 0.00%; FB1: 0.00 36 97 vn: precision: 46.43%; recall: 49.76%; FB1: 48.04 672 98 w: precision: 96.44%; recall: 95.69%; FB1: 96.07 3177 99 xc: precision: 82.48%; recall: 76.35%; FB1: 79.30 137 效果上可能是由于标签太多？另外数据集规模可能也不够大，从参数上这种情况怎么调整可能能取得更好的效果呢？谢谢！

您好，我也在自己的数据集和标签上进行实验，得到的结果全是0。对于数据集，模型输入是每行一个字以及对应的标签，以空格隔开，句子与句子之间用空行隔开，除了这些处理，请问您在自己的数据集上还做了哪些处理呢？

您好，请问您解决了全是0的结果了嘛

同问，全是0怎么解决的？我也是用自己的数据集全是零，目前不知所措。 你好，请问您解决了么，我目前也遇到这个问题