search

syuoni / eznlp

Easy Natural Language Processing
Apache License 2.0
130 stars 21 forks source link

WeiboNER 评估 #29

Closed lingvisa closed 2 years ago

lingvisa commented 2 years ago

您好，我跑了你的代码，没做任何修改，就用公共数据集 Weibo NER 评估，发现指标很低. 命令如下：

python scripts/entity_recognition.py --dataset WeiboNER --ck_decoder boundary_selection

得到的结果只有 40% 多，参数没做任何变动，日志如下：


(eznlp) root@341149:/data3/min/eznlp# python scripts/entity_recognition.py --dataset WeiboNER --ck_decoder boundary_selection
[2022-07-16 10:03:33 INFO] ============================================= Starting =============================================
[2022-07-16 10:03:33 INFO] scripts/entity_recognition.py --dataset WeiboNER --ck_decoder boundary_selection
[2022-07-16 10:03:33 INFO] {'affine_arch': 'FFN',
 'agg_mode': 'max_pooling',
 'batch_size': 64,
 'bert_arch': 'None',
 'bert_drop_rate': 0.2,
 'char_arch': 'None',
 'ck_decoder': 'boundary_selection',
 'ck_size_emb_dim': 25,
 'corrupt_rate': 0.0,
 'dataset': 'WeiboNER',
 'doc_level': False,
 'drop_rate': 0.5,
 'emb_dim': 100,
 'emb_freeze': False,
 'enc_arch': 'LSTM',
 'finetune_lr': 2e-05,
 'fl_gamma': 0.0,
 'grad_clip': 5.0,
 'hard_neg_sampling_rate': 1.0,
 'hard_neg_sampling_size': 5,
 'hid_dim': 200,
 'log_terminal': True,
 'lr': 0.001,
 'max_span_size': 10,
 'neg_sampling_rate': 1.0,
 'num_epochs': 100,
 'num_grad_acc_steps': 1,
 'num_layers': 1,
 'num_neg_chunks': 100,
 'optimizer': 'AdamW',
 'pdb': False,
 'pipeline': False,
 'profile': False,
 'save_preds': False,
 'sb_adj_factor': 1.0,
 'sb_epsilon': 0.0,
 'sb_size': 1,
 'scheduler': 'None',
 'scheme': 'BIOES',
 'seed': 515,
 'sl_epsilon': 0.0,
 'train_with_dev': False,
 'use_amp': False,
 'use_biaffine': True,
 'use_bigram': False,
 'use_crf': True,
 'use_elmo': False,
 'use_flair': False,
 'use_interm1': False,
 'use_interm2': False,
 'use_locked_drop': False,
 'use_softlexicon': False,
 'use_softword': False}
[2022-07-16 10:03:33 INFO] -------------------------------------------- Preparing ---------------------------------------------
[2022-07-16 10:03:33 INFO] Automatically allocating device...
[2022-07-16 10:03:33 INFO] Cuda device `cuda:2` with free memory 28737 MiB successfully allocated, device `cuda:2` returned
[2022-07-16 10:03:37 INFO] No nested chunks detected, only flat chunks are allowed in decoding...
[2022-07-16 10:03:37 INFO] The dataset consists 1,350 sequences
The average `tokens` length is 54.7
The maximum `tokens` length is 175
The dataset has 1,895 chunks of 8 types
[2022-07-16 10:03:37 INFO] --------------------------------------------- Building ---------------------------------------------
[2022-07-16 10:03:37 INFO] Embeddings initialized with randomized vectors 
Vector average absolute value: 0.0866
[2022-07-16 10:03:37 INFO] Embeddings initialized with randomized vectors 
Vector average absolute value: 0.1732
[2022-07-16 10:03:37 INFO] The model has 573,484 parameters, in which 573,484 are trainable and 0 are frozen.
[2022-07-16 10:03:37 INFO] --------------------------------------------- Training ---------------------------------------------
[2022-07-16 10:03:37 INFO] Grouped parameters (573,484) == Model parameters (573,484)
[2022-07-16 10:03:59 INFO] Epoch: 1 | Step: 22 | LR: (0.000020/0.001000)
[2022-07-16 10:03:59 INFO]  Train Loss: 1301.205 | Train Metrics: 0.16% | Elapsed Time: 0m 22s
[2022-07-16 10:04:02 INFO]  Dev.  Loss: 28.731 | Dev.  Metrics: 0.00% | Elapsed Time: 0m 2s
[2022-07-16 10:04:23 INFO] Epoch: 2 | Step: 44 | LR: (0.000020/0.001000)
[2022-07-16 10:04:23 INFO]  Train Loss: 15.072 | Train Metrics: 0.00% | Elapsed Time: 0m 21s
[2022-07-16 10:04:26 INFO]  Dev.  Loss: 17.105 | Dev.  Metrics: 0.00% | Elapsed Time: 0m 2s
[2022-07-16 10:04:47 INFO] Epoch: 3 | Step: 66 | LR: (0.000020/0.001000)
[2022-07-16 10:04:47 INFO]  Train Loss: 11.948 | Train Metrics: 0.00% | Elapsed Time: 0m 20s
[2022-07-16 10:04:50 INFO]  Dev.  Loss: 13.938 | Dev.  Metrics: 0.00% | Elapsed Time: 0m 2s
[2022-07-16 10:05:11 INFO] Epoch: 4 | Step: 88 | LR: (0.000020/0.001000)
[2022-07-16 10:05:11 INFO]  Train Loss: 9.826 | Train Metrics: 0.00% | Elapsed Time: 0m 20s
[2022-07-16 10:05:14 INFO]  Dev.  Loss: 11.808 | Dev.  Metrics: 0.00% | Elapsed Time: 0m 2s
[2022-07-16 10:05:35 INFO] Epoch: 5 | Step: 110 | LR: (0.000020/0.001000)
[2022-07-16 10:05:35 INFO]  Train Loss: 8.653 | Train Metrics: 0.42% | Elapsed Time: 0m 21s
[2022-07-16 10:05:38 INFO]  Dev.  Loss: 10.595 | Dev.  Metrics: 19.86% | Elapsed Time: 0m 2s
[2022-07-16 10:05:59 INFO] Epoch: 6 | Step: 132 | LR: (0.000020/0.001000)
[2022-07-16 10:05:59 INFO]  Train Loss: 7.556 | Train Metrics: 2.19% | Elapsed Time: 0m 21s
[2022-07-16 10:06:02 INFO]  Dev.  Loss: 9.792 | Dev.  Metrics: 35.02% | Elapsed Time: 0m 2s
[2022-07-16 10:06:23 INFO] Epoch: 7 | Step: 154 | LR: (0.000020/0.001000)
[2022-07-16 10:06:23 INFO]  Train Loss: 6.821 | Train Metrics: 7.59% | Elapsed Time: 0m 21s
[2022-07-16 10:06:26 INFO]  Dev.  Loss: 8.919 | Dev.  Metrics: 43.63% | Elapsed Time: 0m 2s
[2022-07-16 10:06:48 INFO] Epoch: 8 | Step: 176 | LR: (0.000020/0.001000)
[2022-07-16 10:06:48 INFO]  Train Loss: 6.231 | Train Metrics: 14.56% | Elapsed Time: 0m 21s
[2022-07-16 10:06:50 INFO]  Dev.  Loss: 8.451 | Dev.  Metrics: 47.48% | Elapsed Time: 0m 2s
[2022-07-16 10:07:11 INFO] Epoch: 9 | Step: 198 | LR: (0.000020/0.001000)
[2022-07-16 10:07:11 INFO]  Train Loss: 5.949 | Train Metrics: 22.35% | Elapsed Time: 0m 20s
[2022-07-16 10:07:14 INFO]  Dev.  Loss: 8.130 | Dev.  Metrics: 47.00% | Elapsed Time: 0m 2s
[2022-07-16 10:07:35 INFO] Epoch: 10 | Step: 220 | LR: (0.000020/0.001000)
[2022-07-16 10:07:35 INFO]  Train Loss: 5.569 | Train Metrics: 25.16% | Elapsed Time: 0m 21s
[2022-07-16 10:07:38 INFO]  Dev.  Loss: 7.772 | Dev.  Metrics: 49.85% | Elapsed Time: 0m 2s
[2022-07-16 10:07:59 INFO] Epoch: 11 | Step: 242 | LR: (0.000020/0.001000)
[2022-07-16 10:07:59 INFO]  Train Loss: 5.346 | Train Metrics: 25.99% | Elapsed Time: 0m 21s
[2022-07-16 10:08:02 INFO]  Dev.  Loss: 7.443 | Dev.  Metrics: 51.42% | Elapsed Time: 0m 2s
[2022-07-16 10:08:23 INFO] Epoch: 12 | Step: 264 | LR: (0.000020/0.001000)
[2022-07-16 10:08:23 INFO]  Train Loss: 5.321 | Train Metrics: 28.73% | Elapsed Time: 0m 21s
[2022-07-16 10:08:26 INFO]  Dev.  Loss: 7.445 | Dev.  Metrics: 50.94% | Elapsed Time: 0m 2s
[2022-07-16 10:08:48 INFO] Epoch: 13 | Step: 286 | LR: (0.000020/0.001000)
[2022-07-16 10:08:48 INFO]  Train Loss: 4.802 | Train Metrics: 33.44% | Elapsed Time: 0m 21s
[2022-07-16 10:08:51 INFO]  Dev.  Loss: 7.038 | Dev.  Metrics: 51.24% | Elapsed Time: 0m 2s
[2022-07-16 10:09:12 INFO] Epoch: 14 | Step: 308 | LR: (0.000020/0.001000)
[2022-07-16 10:09:12 INFO]  Train Loss: 4.676 | Train Metrics: 36.31% | Elapsed Time: 0m 20s
[2022-07-16 10:09:15 INFO]  Dev.  Loss: 7.063 | Dev.  Metrics: 51.59% | Elapsed Time: 0m 2s
[2022-07-16 10:09:36 INFO] Epoch: 15 | Step: 330 | LR: (0.000020/0.001000)
[2022-07-16 10:09:36 INFO]  Train Loss: 4.749 | Train Metrics: 38.52% | Elapsed Time: 0m 20s
[2022-07-16 10:09:38 INFO]  Dev.  Loss: 6.929 | Dev.  Metrics: 51.86% | Elapsed Time: 0m 2s
[2022-07-16 10:10:00 INFO] Epoch: 16 | Step: 352 | LR: (0.000020/0.001000)
[2022-07-16 10:10:00 INFO]  Train Loss: 4.670 | Train Metrics: 40.03% | Elapsed Time: 0m 21s
[2022-07-16 10:10:03 INFO]  Dev.  Loss: 6.693 | Dev.  Metrics: 53.56% | Elapsed Time: 0m 2s
[2022-07-16 10:10:24 INFO] Epoch: 17 | Step: 374 | LR: (0.000020/0.001000)
[2022-07-16 10:10:24 INFO]  Train Loss: 4.484 | Train Metrics: 41.32% | Elapsed Time: 0m 21s
[2022-07-16 10:10:27 INFO]  Dev.  Loss: 6.942 | Dev.  Metrics: 53.20% | Elapsed Time: 0m 3s
[2022-07-16 10:10:48 INFO] Epoch: 18 | Step: 396 | LR: (0.000020/0.001000)
[2022-07-16 10:10:48 INFO]  Train Loss: 4.276 | Train Metrics: 43.86% | Elapsed Time: 0m 20s
[2022-07-16 10:10:51 INFO]  Dev.  Loss: 6.542 | Dev.  Metrics: 53.64% | Elapsed Time: 0m 3s
[2022-07-16 10:11:12 INFO] Epoch: 19 | Step: 418 | LR: (0.000020/0.001000)
[2022-07-16 10:11:12 INFO]  Train Loss: 4.042 | Train Metrics: 43.79% | Elapsed Time: 0m 21s
[2022-07-16 10:11:15 INFO]  Dev.  Loss: 6.566 | Dev.  Metrics: 53.28% | Elapsed Time: 0m 2s
[2022-07-16 10:11:36 INFO] Epoch: 20 | Step: 440 | LR: (0.000020/0.001000)
[2022-07-16 10:11:36 INFO]  Train Loss: 3.863 | Train Metrics: 45.25% | Elapsed Time: 0m 20s
[2022-07-16 10:11:39 INFO]  Dev.  Loss: 6.259 | Dev.  Metrics: 54.21% | Elapsed Time: 0m 2s
[2022-07-16 10:12:00 INFO] Epoch: 21 | Step: 462 | LR: (0.000020/0.001000)
[2022-07-16 10:12:00 INFO]  Train Loss: 3.860 | Train Metrics: 47.51% | Elapsed Time: 0m 21s
[2022-07-16 10:12:03 INFO]  Dev.  Loss: 6.482 | Dev.  Metrics: 54.20% | Elapsed Time: 0m 2s
[2022-07-16 10:12:24 INFO] Epoch: 22 | Step: 484 | LR: (0.000020/0.001000)
[2022-07-16 10:12:24 INFO]  Train Loss: 3.939 | Train Metrics: 49.15% | Elapsed Time: 0m 21s
[2022-07-16 10:12:27 INFO]  Dev.  Loss: 6.268 | Dev.  Metrics: 54.60% | Elapsed Time: 0m 2s
[2022-07-16 10:12:48 INFO] Epoch: 23 | Step: 506 | LR: (0.000020/0.001000)
[2022-07-16 10:12:48 INFO]  Train Loss: 3.862 | Train Metrics: 49.88% | Elapsed Time: 0m 21s
[2022-07-16 10:12:51 INFO]  Dev.  Loss: 6.171 | Dev.  Metrics: 54.83% | Elapsed Time: 0m 2s
[2022-07-16 10:13:12 INFO] Epoch: 24 | Step: 528 | LR: (0.000020/0.001000)
[2022-07-16 10:13:12 INFO]  Train Loss: 3.753 | Train Metrics: 48.67% | Elapsed Time: 0m 20s
[2022-07-16 10:13:15 INFO]  Dev.  Loss: 6.303 | Dev.  Metrics: 54.37% | Elapsed Time: 0m 2s
[2022-07-16 10:13:36 INFO] Epoch: 25 | Step: 550 | LR: (0.000020/0.001000)
[2022-07-16 10:13:36 INFO]  Train Loss: 3.477 | Train Metrics: 48.90% | Elapsed Time: 0m 21s
[2022-07-16 10:13:39 INFO]  Dev.  Loss: 6.153 | Dev.  Metrics: 55.25% | Elapsed Time: 0m 2s
[2022-07-16 10:14:00 INFO] Epoch: 26 | Step: 572 | LR: (0.000020/0.001000)
[2022-07-16 10:14:00 INFO]  Train Loss: 3.443 | Train Metrics: 49.56% | Elapsed Time: 0m 21s
[2022-07-16 10:14:03 INFO]  Dev.  Loss: 6.249 | Dev.  Metrics: 53.99% | Elapsed Time: 0m 2s
[2022-07-16 10:14:24 INFO] Epoch: 27 | Step: 594 | LR: (0.000020/0.001000)
[2022-07-16 10:14:24 INFO]  Train Loss: 3.445 | Train Metrics: 51.67% | Elapsed Time: 0m 21s
[2022-07-16 10:14:27 INFO]  Dev.  Loss: 6.230 | Dev.  Metrics: 54.82% | Elapsed Time: 0m 2s
[2022-07-16 10:14:48 INFO] Epoch: 28 | Step: 616 | LR: (0.000020/0.001000)
[2022-07-16 10:14:48 INFO]  Train Loss: 3.573 | Train Metrics: 52.76% | Elapsed Time: 0m 20s
[2022-07-16 10:14:51 INFO]  Dev.  Loss: 6.234 | Dev.  Metrics: 55.97% | Elapsed Time: 0m 2s
[2022-07-16 10:15:12 INFO] Epoch: 29 | Step: 638 | LR: (0.000020/0.001000)
[2022-07-16 10:15:12 INFO]  Train Loss: 3.328 | Train Metrics: 55.12% | Elapsed Time: 0m 21s
[2022-07-16 10:15:15 INFO]  Dev.  Loss: 6.255 | Dev.  Metrics: 55.63% | Elapsed Time: 0m 2s
[2022-07-16 10:15:35 INFO] Epoch: 30 | Step: 660 | LR: (0.000020/0.001000)
[2022-07-16 10:15:35 INFO]  Train Loss: 3.313 | Train Metrics: 53.71% | Elapsed Time: 0m 20s
[2022-07-16 10:15:38 INFO]  Dev.  Loss: 6.213 | Dev.  Metrics: 55.51% | Elapsed Time: 0m 2s
[2022-07-16 10:16:00 INFO] Epoch: 31 | Step: 682 | LR: (0.000020/0.001000)
[2022-07-16 10:16:00 INFO]  Train Loss: 3.158 | Train Metrics: 55.92% | Elapsed Time: 0m 21s
[2022-07-16 10:16:02 INFO]  Dev.  Loss: 6.180 | Dev.  Metrics: 55.45% | Elapsed Time: 0m 2s
[2022-07-16 10:16:23 INFO] Epoch: 32 | Step: 704 | LR: (0.000020/0.001000)
[2022-07-16 10:16:23 INFO]  Train Loss: 3.223 | Train Metrics: 56.11% | Elapsed Time: 0m 21s
[2022-07-16 10:16:26 INFO]  Dev.  Loss: 6.209 | Dev.  Metrics: 56.02% | Elapsed Time: 0m 2s
[2022-07-16 10:16:47 INFO] Epoch: 33 | Step: 726 | LR: (0.000020/0.001000)
[2022-07-16 10:16:47 INFO]  Train Loss: 3.114 | Train Metrics: 56.55% | Elapsed Time: 0m 20s
[2022-07-16 10:16:50 INFO]  Dev.  Loss: 6.277 | Dev.  Metrics: 55.26% | Elapsed Time: 0m 3s
[2022-07-16 10:17:12 INFO] Epoch: 34 | Step: 748 | LR: (0.000020/0.001000)
[2022-07-16 10:17:12 INFO]  Train Loss: 2.960 | Train Metrics: 57.34% | Elapsed Time: 0m 21s
[2022-07-16 10:17:14 INFO]  Dev.  Loss: 6.176 | Dev.  Metrics: 55.03% | Elapsed Time: 0m 2s
[2022-07-16 10:17:35 INFO] Epoch: 35 | Step: 770 | LR: (0.000020/0.001000)
[2022-07-16 10:17:35 INFO]  Train Loss: 3.043 | Train Metrics: 56.51% | Elapsed Time: 0m 20s
[2022-07-16 10:17:38 INFO]  Dev.  Loss: 6.106 | Dev.  Metrics: 56.52% | Elapsed Time: 0m 2s
[2022-07-16 10:18:00 INFO] Epoch: 36 | Step: 792 | LR: (0.000020/0.001000)
[2022-07-16 10:18:00 INFO]  Train Loss: 2.964 | Train Metrics: 56.14% | Elapsed Time: 0m 21s
[2022-07-16 10:18:02 INFO]  Dev.  Loss: 6.239 | Dev.  Metrics: 56.06% | Elapsed Time: 0m 2s
[2022-07-16 10:18:24 INFO] Epoch: 37 | Step: 814 | LR: (0.000020/0.001000)
[2022-07-16 10:18:24 INFO]  Train Loss: 3.106 | Train Metrics: 57.23% | Elapsed Time: 0m 21s
[2022-07-16 10:18:27 INFO]  Dev.  Loss: 6.295 | Dev.  Metrics: 56.07% | Elapsed Time: 0m 2s
[2022-07-16 10:18:48 INFO] Epoch: 38 | Step: 836 | LR: (0.000020/0.001000)
[2022-07-16 10:18:48 INFO]  Train Loss: 2.903 | Train Metrics: 59.61% | Elapsed Time: 0m 21s
[2022-07-16 10:18:51 INFO]  Dev.  Loss: 6.185 | Dev.  Metrics: 56.91% | Elapsed Time: 0m 2s
[2022-07-16 10:19:11 INFO] Epoch: 39 | Step: 858 | LR: (0.000020/0.001000)
[2022-07-16 10:19:11 INFO]  Train Loss: 2.865 | Train Metrics: 59.19% | Elapsed Time: 0m 20s
[2022-07-16 10:19:14 INFO]  Dev.  Loss: 6.312 | Dev.  Metrics: 56.49% | Elapsed Time: 0m 2s
[2022-07-16 10:19:35 INFO] Epoch: 40 | Step: 880 | LR: (0.000020/0.001000)
[2022-07-16 10:19:35 INFO]  Train Loss: 2.830 | Train Metrics: 60.04% | Elapsed Time: 0m 21s
[2022-07-16 10:19:38 INFO]  Dev.  Loss: 6.221 | Dev.  Metrics: 57.10% | Elapsed Time: 0m 2s
[2022-07-16 10:19:59 INFO] Epoch: 41 | Step: 902 | LR: (0.000020/0.001000)
[2022-07-16 10:19:59 INFO]  Train Loss: 2.834 | Train Metrics: 59.83% | Elapsed Time: 0m 21s
[2022-07-16 10:20:02 INFO]  Dev.  Loss: 6.211 | Dev.  Metrics: 56.99% | Elapsed Time: 0m 2s
[2022-07-16 10:20:23 INFO] Epoch: 42 | Step: 924 | LR: (0.000020/0.001000)
[2022-07-16 10:20:23 INFO]  Train Loss: 2.842 | Train Metrics: 59.99% | Elapsed Time: 0m 20s
[2022-07-16 10:20:26 INFO]  Dev.  Loss: 6.235 | Dev.  Metrics: 56.00% | Elapsed Time: 0m 2s
[2022-07-16 10:20:47 INFO] Epoch: 43 | Step: 946 | LR: (0.000020/0.001000)
[2022-07-16 10:20:47 INFO]  Train Loss: 2.693 | Train Metrics: 61.06% | Elapsed Time: 0m 21s
[2022-07-16 10:20:50 INFO]  Dev.  Loss: 6.214 | Dev.  Metrics: 56.48% | Elapsed Time: 0m 2s
[2022-07-16 10:21:11 INFO] Epoch: 44 | Step: 968 | LR: (0.000020/0.001000)
[2022-07-16 10:21:11 INFO]  Train Loss: 2.590 | Train Metrics: 61.71% | Elapsed Time: 0m 21s
[2022-07-16 10:21:14 INFO]  Dev.  Loss: 6.192 | Dev.  Metrics: 56.55% | Elapsed Time: 0m 2s
[2022-07-16 10:21:35 INFO] Epoch: 45 | Step: 990 | LR: (0.000020/0.001000)
[2022-07-16 10:21:35 INFO]  Train Loss: 2.672 | Train Metrics: 60.78% | Elapsed Time: 0m 21s
[2022-07-16 10:21:38 INFO]  Dev.  Loss: 6.269 | Dev.  Metrics: 55.63% | Elapsed Time: 0m 2s
[2022-07-16 10:21:59 INFO] Epoch: 46 | Step: 1012 | LR: (0.000020/0.001000)
[2022-07-16 10:21:59 INFO]  Train Loss: 2.705 | Train Metrics: 61.97% | Elapsed Time: 0m 21s
[2022-07-16 10:22:02 INFO]  Dev.  Loss: 6.377 | Dev.  Metrics: 56.11% | Elapsed Time: 0m 2s
[2022-07-16 10:22:23 INFO] Epoch: 47 | Step: 1034 | LR: (0.000020/0.001000)
[2022-07-16 10:22:23 INFO]  Train Loss: 2.740 | Train Metrics: 63.14% | Elapsed Time: 0m 20s
[2022-07-16 10:22:26 INFO]  Dev.  Loss: 6.294 | Dev.  Metrics: 56.55% | Elapsed Time: 0m 2s
[2022-07-16 10:22:47 INFO] Epoch: 48 | Step: 1056 | LR: (0.000020/0.001000)
[2022-07-16 10:22:47 INFO]  Train Loss: 2.670 | Train Metrics: 63.13% | Elapsed Time: 0m 21s
[2022-07-16 10:22:50 INFO]  Dev.  Loss: 6.371 | Dev.  Metrics: 56.56% | Elapsed Time: 0m 2s
[2022-07-16 10:23:11 INFO] Epoch: 49 | Step: 1078 | LR: (0.000020/0.001000)
[2022-07-16 10:23:11 INFO]  Train Loss: 2.502 | Train Metrics: 63.05% | Elapsed Time: 0m 20s
[2022-07-16 10:23:14 INFO]  Dev.  Loss: 6.335 | Dev.  Metrics: 56.91% | Elapsed Time: 0m 2s
[2022-07-16 10:23:35 INFO] Epoch: 50 | Step: 1100 | LR: (0.000020/0.001000)
[2022-07-16 10:23:35 INFO]  Train Loss: 2.510 | Train Metrics: 63.35% | Elapsed Time: 0m 21s
[2022-07-16 10:23:38 INFO]  Dev.  Loss: 6.433 | Dev.  Metrics: 56.42% | Elapsed Time: 0m 2s
[2022-07-16 10:23:59 INFO] Epoch: 51 | Step: 1122 | LR: (0.000020/0.001000)
[2022-07-16 10:23:59 INFO]  Train Loss: 2.498 | Train Metrics: 62.77% | Elapsed Time: 0m 21s
[2022-07-16 10:24:02 INFO]  Dev.  Loss: 6.462 | Dev.  Metrics: 56.16% | Elapsed Time: 0m 2s
[2022-07-16 10:24:23 INFO] Epoch: 52 | Step: 1144 | LR: (0.000020/0.001000)
[2022-07-16 10:24:23 INFO]  Train Loss: 2.385 | Train Metrics: 65.18% | Elapsed Time: 0m 21s
[2022-07-16 10:24:26 INFO]  Dev.  Loss: 6.312 | Dev.  Metrics: 56.17% | Elapsed Time: 0m 2s
[2022-07-16 10:24:48 INFO] Epoch: 53 | Step: 1166 | LR: (0.000020/0.001000)
[2022-07-16 10:24:48 INFO]  Train Loss: 2.470 | Train Metrics: 63.45% | Elapsed Time: 0m 21s
[2022-07-16 10:24:50 INFO]  Dev.  Loss: 6.366 | Dev.  Metrics: 56.56% | Elapsed Time: 0m 2s
[2022-07-16 10:25:11 INFO] Epoch: 54 | Step: 1188 | LR: (0.000020/0.001000)
[2022-07-16 10:25:11 INFO]  Train Loss: 2.436 | Train Metrics: 65.99% | Elapsed Time: 0m 20s
[2022-07-16 10:25:14 INFO]  Dev.  Loss: 6.447 | Dev.  Metrics: 55.93% | Elapsed Time: 0m 2s
[2022-07-16 10:25:36 INFO] Epoch: 55 | Step: 1210 | LR: (0.000020/0.001000)
[2022-07-16 10:25:36 INFO]  Train Loss: 2.386 | Train Metrics: 65.98% | Elapsed Time: 0m 21s
[2022-07-16 10:25:38 INFO]  Dev.  Loss: 6.478 | Dev.  Metrics: 56.84% | Elapsed Time: 0m 2s
[2022-07-16 10:26:00 INFO] Epoch: 56 | Step: 1232 | LR: (0.000020/0.001000)
[2022-07-16 10:26:00 INFO]  Train Loss: 2.431 | Train Metrics: 63.24% | Elapsed Time: 0m 21s
[2022-07-16 10:26:02 INFO]  Dev.  Loss: 6.480 | Dev.  Metrics: 56.27% | Elapsed Time: 0m 2s
[2022-07-16 10:26:24 INFO] Epoch: 57 | Step: 1254 | LR: (0.000020/0.001000)
[2022-07-16 10:26:24 INFO]  Train Loss: 2.452 | Train Metrics: 66.91% | Elapsed Time: 0m 21s
[2022-07-16 10:26:27 INFO]  Dev.  Loss: 6.421 | Dev.  Metrics: 56.55% | Elapsed Time: 0m 2s
[2022-07-16 10:26:48 INFO] Epoch: 58 | Step: 1276 | LR: (0.000020/0.001000)
[2022-07-16 10:26:48 INFO]  Train Loss: 2.371 | Train Metrics: 65.19% | Elapsed Time: 0m 21s
[2022-07-16 10:26:51 INFO]  Dev.  Loss: 6.527 | Dev.  Metrics: 56.02% | Elapsed Time: 0m 2s
[2022-07-16 10:27:12 INFO] Epoch: 59 | Step: 1298 | LR: (0.000020/0.001000)
[2022-07-16 10:27:12 INFO]  Train Loss: 2.356 | Train Metrics: 68.34% | Elapsed Time: 0m 21s
[2022-07-16 10:27:15 INFO]  Dev.  Loss: 6.513 | Dev.  Metrics: 56.45% | Elapsed Time: 0m 2s
[2022-07-16 10:27:36 INFO] Epoch: 60 | Step: 1320 | LR: (0.000020/0.001000)
[2022-07-16 10:27:36 INFO]  Train Loss: 2.371 | Train Metrics: 66.42% | Elapsed Time: 0m 20s
[2022-07-16 10:27:39 INFO]  Dev.  Loss: 6.597 | Dev.  Metrics: 56.42% | Elapsed Time: 0m 2s
[2022-07-16 10:27:59 INFO] Epoch: 61 | Step: 1342 | LR: (0.000020/0.001000)
[2022-07-16 10:27:59 INFO]  Train Loss: 2.474 | Train Metrics: 65.50% | Elapsed Time: 0m 20s
[2022-07-16 10:28:02 INFO]  Dev.  Loss: 6.492 | Dev.  Metrics: 57.06% | Elapsed Time: 0m 2s
[2022-07-16 10:28:23 INFO] Epoch: 62 | Step: 1364 | LR: (0.000020/0.001000)
[2022-07-16 10:28:23 INFO]  Train Loss: 2.238 | Train Metrics: 66.65% | Elapsed Time: 0m 20s
[2022-07-16 10:28:26 INFO]  Dev.  Loss: 6.514 | Dev.  Metrics: 57.42% | Elapsed Time: 0m 2s
[2022-07-16 10:28:48 INFO] Epoch: 63 | Step: 1386 | LR: (0.000020/0.001000)
[2022-07-16 10:28:48 INFO]  Train Loss: 2.465 | Train Metrics: 68.83% | Elapsed Time: 0m 21s
[2022-07-16 10:28:51 INFO]  Dev.  Loss: 6.601 | Dev.  Metrics: 57.10% | Elapsed Time: 0m 2s
[2022-07-16 10:29:12 INFO] Epoch: 64 | Step: 1408 | LR: (0.000020/0.001000)
[2022-07-16 10:29:12 INFO]  Train Loss: 2.205 | Train Metrics: 67.29% | Elapsed Time: 0m 21s
[2022-07-16 10:29:15 INFO]  Dev.  Loss: 6.528 | Dev.  Metrics: 57.06% | Elapsed Time: 0m 2s
[2022-07-16 10:29:36 INFO] Epoch: 65 | Step: 1430 | LR: (0.000020/0.001000)
[2022-07-16 10:29:36 INFO]  Train Loss: 2.245 | Train Metrics: 65.63% | Elapsed Time: 0m 21s
[2022-07-16 10:29:39 INFO]  Dev.  Loss: 6.595 | Dev.  Metrics: 56.95% | Elapsed Time: 0m 2s
[2022-07-16 10:30:00 INFO] Epoch: 66 | Step: 1452 | LR: (0.000020/0.001000)
[2022-07-16 10:30:00 INFO]  Train Loss: 2.277 | Train Metrics: 67.15% | Elapsed Time: 0m 20s
[2022-07-16 10:30:03 INFO]  Dev.  Loss: 6.653 | Dev.  Metrics: 57.38% | Elapsed Time: 0m 2s
[2022-07-16 10:30:24 INFO] Epoch: 67 | Step: 1474 | LR: (0.000020/0.001000)
[2022-07-16 10:30:24 INFO]  Train Loss: 2.320 | Train Metrics: 67.60% | Elapsed Time: 0m 21s
[2022-07-16 10:30:27 INFO]  Dev.  Loss: 6.530 | Dev.  Metrics: 56.99% | Elapsed Time: 0m 2s
[2022-07-16 10:30:49 INFO] Epoch: 68 | Step: 1496 | LR: (0.000020/0.001000)
[2022-07-16 10:30:49 INFO]  Train Loss: 2.272 | Train Metrics: 68.62% | Elapsed Time: 0m 21s
[2022-07-16 10:30:51 INFO]  Dev.  Loss: 6.672 | Dev.  Metrics: 56.64% | Elapsed Time: 0m 2s
[2022-07-16 10:31:13 INFO] Epoch: 69 | Step: 1518 | LR: (0.000020/0.001000)
[2022-07-16 10:31:13 INFO]  Train Loss: 2.197 | Train Metrics: 70.27% | Elapsed Time: 0m 21s
[2022-07-16 10:31:16 INFO]  Dev.  Loss: 6.659 | Dev.  Metrics: 56.91% | Elapsed Time: 0m 2s
[2022-07-16 10:31:36 INFO] Epoch: 70 | Step: 1540 | LR: (0.000020/0.001000)
[2022-07-16 10:31:36 INFO]  Train Loss: 2.333 | Train Metrics: 68.11% | Elapsed Time: 0m 20s
[2022-07-16 10:31:39 INFO]  Dev.  Loss: 6.881 | Dev.  Metrics: 56.88% | Elapsed Time: 0m 2s
[2022-07-16 10:32:00 INFO] Epoch: 71 | Step: 1562 | LR: (0.000020/0.001000)
[2022-07-16 10:32:00 INFO]  Train Loss: 2.146 | Train Metrics: 68.20% | Elapsed Time: 0m 21s
[2022-07-16 10:32:03 INFO]  Dev.  Loss: 6.757 | Dev.  Metrics: 56.83% | Elapsed Time: 0m 2s
[2022-07-16 10:32:25 INFO] Epoch: 72 | Step: 1584 | LR: (0.000020/0.001000)
[2022-07-16 10:32:25 INFO]  Train Loss: 2.102 | Train Metrics: 69.26% | Elapsed Time: 0m 21s
[2022-07-16 10:32:28 INFO]  Dev.  Loss: 6.784 | Dev.  Metrics: 57.22% | Elapsed Time: 0m 2s
[2022-07-16 10:32:49 INFO] Epoch: 73 | Step: 1606 | LR: (0.000020/0.001000)
[2022-07-16 10:32:49 INFO]  Train Loss: 2.229 | Train Metrics: 69.22% | Elapsed Time: 0m 21s
[2022-07-16 10:32:52 INFO]  Dev.  Loss: 6.727 | Dev.  Metrics: 57.14% | Elapsed Time: 0m 2s
[2022-07-16 10:33:13 INFO] Epoch: 74 | Step: 1628 | LR: (0.000020/0.001000)
[2022-07-16 10:33:13 INFO]  Train Loss: 2.092 | Train Metrics: 69.13% | Elapsed Time: 0m 21s
[2022-07-16 10:33:16 INFO]  Dev.  Loss: 6.760 | Dev.  Metrics: 57.65% | Elapsed Time: 0m 2s
[2022-07-16 10:33:37 INFO] Epoch: 75 | Step: 1650 | LR: (0.000020/0.001000)
[2022-07-16 10:33:37 INFO]  Train Loss: 2.120 | Train Metrics: 70.61% | Elapsed Time: 0m 21s
[2022-07-16 10:33:40 INFO]  Dev.  Loss: 6.841 | Dev.  Metrics: 56.91% | Elapsed Time: 0m 2s
[2022-07-16 10:34:01 INFO] Epoch: 76 | Step: 1672 | LR: (0.000020/0.001000)
[2022-07-16 10:34:01 INFO]  Train Loss: 2.128 | Train Metrics: 70.90% | Elapsed Time: 0m 20s
[2022-07-16 10:34:04 INFO]  Dev.  Loss: 6.822 | Dev.  Metrics: 56.40% | Elapsed Time: 0m 2s
[2022-07-16 10:34:25 INFO] Epoch: 77 | Step: 1694 | LR: (0.000020/0.001000)
[2022-07-16 10:34:25 INFO]  Train Loss: 2.261 | Train Metrics: 69.97% | Elapsed Time: 0m 21s
[2022-07-16 10:34:28 INFO]  Dev.  Loss: 6.827 | Dev.  Metrics: 56.43% | Elapsed Time: 0m 2s
[2022-07-16 10:34:48 INFO] Epoch: 78 | Step: 1716 | LR: (0.000020/0.001000)
[2022-07-16 10:34:48 INFO]  Train Loss: 2.051 | Train Metrics: 71.30% | Elapsed Time: 0m 20s
[2022-07-16 10:34:51 INFO]  Dev.  Loss: 6.900 | Dev.  Metrics: 56.60% | Elapsed Time: 0m 2s
[2022-07-16 10:35:12 INFO] Epoch: 79 | Step: 1738 | LR: (0.000020/0.001000)
[2022-07-16 10:35:12 INFO]  Train Loss: 2.042 | Train Metrics: 72.14% | Elapsed Time: 0m 21s
[2022-07-16 10:35:15 INFO]  Dev.  Loss: 6.915 | Dev.  Metrics: 55.04% | Elapsed Time: 0m 2s
[2022-07-16 10:35:36 INFO] Epoch: 80 | Step: 1760 | LR: (0.000020/0.001000)
[2022-07-16 10:35:36 INFO]  Train Loss: 2.104 | Train Metrics: 72.53% | Elapsed Time: 0m 21s
[2022-07-16 10:35:39 INFO]  Dev.  Loss: 6.937 | Dev.  Metrics: 55.42% | Elapsed Time: 0m 2s
[2022-07-16 10:36:00 INFO] Epoch: 81 | Step: 1782 | LR: (0.000020/0.001000)
[2022-07-16 10:36:00 INFO]  Train Loss: 2.054 | Train Metrics: 70.77% | Elapsed Time: 0m 21s
[2022-07-16 10:36:03 INFO]  Dev.  Loss: 6.890 | Dev.  Metrics: 55.69% | Elapsed Time: 0m 2s
[2022-07-16 10:36:25 INFO] Epoch: 82 | Step: 1804 | LR: (0.000020/0.001000)
[2022-07-16 10:36:25 INFO]  Train Loss: 2.008 | Train Metrics: 70.87% | Elapsed Time: 0m 21s
[2022-07-16 10:36:28 INFO]  Dev.  Loss: 6.916 | Dev.  Metrics: 55.63% | Elapsed Time: 0m 2s
[2022-07-16 10:36:49 INFO] Epoch: 83 | Step: 1826 | LR: (0.000020/0.001000)
[2022-07-16 10:36:49 INFO]  Train Loss: 2.021 | Train Metrics: 71.09% | Elapsed Time: 0m 21s
[2022-07-16 10:36:52 INFO]  Dev.  Loss: 7.085 | Dev.  Metrics: 54.52% | Elapsed Time: 0m 2s
[2022-07-16 10:37:13 INFO] Epoch: 84 | Step: 1848 | LR: (0.000020/0.001000)
[2022-07-16 10:37:13 INFO]  Train Loss: 2.032 | Train Metrics: 73.35% | Elapsed Time: 0m 21s
[2022-07-16 10:37:16 INFO]  Dev.  Loss: 7.043 | Dev.  Metrics: 54.91% | Elapsed Time: 0m 2s
[2022-07-16 10:37:37 INFO] Epoch: 85 | Step: 1870 | LR: (0.000020/0.001000)
[2022-07-16 10:37:37 INFO]  Train Loss: 2.176 | Train Metrics: 72.74% | Elapsed Time: 0m 21s
[2022-07-16 10:37:40 INFO]  Dev.  Loss: 6.886 | Dev.  Metrics: 56.19% | Elapsed Time: 0m 2s
[2022-07-16 10:38:01 INFO] Epoch: 86 | Step: 1892 | LR: (0.000020/0.001000)
[2022-07-16 10:38:01 INFO]  Train Loss: 1.920 | Train Metrics: 72.71% | Elapsed Time: 0m 21s
[2022-07-16 10:38:04 INFO]  Dev.  Loss: 6.930 | Dev.  Metrics: 55.85% | Elapsed Time: 0m 2s
[2022-07-16 10:38:25 INFO] Epoch: 87 | Step: 1914 | LR: (0.000020/0.001000)
[2022-07-16 10:38:25 INFO]  Train Loss: 1.911 | Train Metrics: 72.68% | Elapsed Time: 0m 21s
[2022-07-16 10:38:28 INFO]  Dev.  Loss: 6.982 | Dev.  Metrics: 56.24% | Elapsed Time: 0m 2s
[2022-07-16 10:38:49 INFO] Epoch: 88 | Step: 1936 | LR: (0.000020/0.001000)
[2022-07-16 10:38:49 INFO]  Train Loss: 2.043 | Train Metrics: 73.03% | Elapsed Time: 0m 21s
[2022-07-16 10:38:52 INFO]  Dev.  Loss: 7.019 | Dev.  Metrics: 55.97% | Elapsed Time: 0m 2s
[2022-07-16 10:39:14 INFO] Epoch: 89 | Step: 1958 | LR: (0.000020/0.001000)
[2022-07-16 10:39:14 INFO]  Train Loss: 1.954 | Train Metrics: 73.31% | Elapsed Time: 0m 21s
[2022-07-16 10:39:17 INFO]  Dev.  Loss: 7.031 | Dev.  Metrics: 56.20% | Elapsed Time: 0m 3s
[2022-07-16 10:39:38 INFO] Epoch: 90 | Step: 1980 | LR: (0.000020/0.001000)
[2022-07-16 10:39:38 INFO]  Train Loss: 1.947 | Train Metrics: 72.35% | Elapsed Time: 0m 21s
[2022-07-16 10:39:41 INFO]  Dev.  Loss: 7.067 | Dev.  Metrics: 55.75% | Elapsed Time: 0m 2s
[2022-07-16 10:40:02 INFO] Epoch: 91 | Step: 2002 | LR: (0.000020/0.001000)
[2022-07-16 10:40:02 INFO]  Train Loss: 2.131 | Train Metrics: 73.43% | Elapsed Time: 0m 20s
[2022-07-16 10:40:04 INFO]  Dev.  Loss: 7.096 | Dev.  Metrics: 56.24% | Elapsed Time: 0m 2s
[2022-07-16 10:40:26 INFO] Epoch: 92 | Step: 2024 | LR: (0.000020/0.001000)
[2022-07-16 10:40:26 INFO]  Train Loss: 1.896 | Train Metrics: 73.53% | Elapsed Time: 0m 21s
[2022-07-16 10:40:29 INFO]  Dev.  Loss: 7.108 | Dev.  Metrics: 55.46% | Elapsed Time: 0m 2s
[2022-07-16 10:40:50 INFO] Epoch: 93 | Step: 2046 | LR: (0.000020/0.001000)
[2022-07-16 10:40:50 INFO]  Train Loss: 2.044 | Train Metrics: 74.36% | Elapsed Time: 0m 20s
[2022-07-16 10:40:53 INFO]  Dev.  Loss: 7.164 | Dev.  Metrics: 55.38% | Elapsed Time: 0m 2s
[2022-07-16 10:41:14 INFO] Epoch: 94 | Step: 2068 | LR: (0.000020/0.001000)
[2022-07-16 10:41:14 INFO]  Train Loss: 1.923 | Train Metrics: 73.56% | Elapsed Time: 0m 21s
[2022-07-16 10:41:17 INFO]  Dev.  Loss: 7.083 | Dev.  Metrics: 56.47% | Elapsed Time: 0m 2s
[2022-07-16 10:41:38 INFO] Epoch: 95 | Step: 2090 | LR: (0.000020/0.001000)
[2022-07-16 10:41:38 INFO]  Train Loss: 1.827 | Train Metrics: 73.45% | Elapsed Time: 0m 21s
[2022-07-16 10:41:41 INFO]  Dev.  Loss: 7.104 | Dev.  Metrics: 56.17% | Elapsed Time: 0m 2s
[2022-07-16 10:42:02 INFO] Epoch: 96 | Step: 2112 | LR: (0.000020/0.001000)
[2022-07-16 10:42:02 INFO]  Train Loss: 1.849 | Train Metrics: 73.83% | Elapsed Time: 0m 21s
[2022-07-16 10:42:05 INFO]  Dev.  Loss: 7.202 | Dev.  Metrics: 55.13% | Elapsed Time: 0m 2s
[2022-07-16 10:42:27 INFO] Epoch: 97 | Step: 2134 | LR: (0.000020/0.001000)
[2022-07-16 10:42:27 INFO]  Train Loss: 1.930 | Train Metrics: 72.34% | Elapsed Time: 0m 21s
[2022-07-16 10:42:30 INFO]  Dev.  Loss: 7.176 | Dev.  Metrics: 56.25% | Elapsed Time: 0m 2s
[2022-07-16 10:42:51 INFO] Epoch: 98 | Step: 2156 | LR: (0.000020/0.001000)
[2022-07-16 10:42:51 INFO]  Train Loss: 2.010 | Train Metrics: 74.25% | Elapsed Time: 0m 20s
[2022-07-16 10:42:53 INFO]  Dev.  Loss: 7.156 | Dev.  Metrics: 56.37% | Elapsed Time: 0m 2s
[2022-07-16 10:43:15 INFO] Epoch: 99 | Step: 2178 | LR: (0.000020/0.001000)
[2022-07-16 10:43:15 INFO]  Train Loss: 1.966 | Train Metrics: 72.51% | Elapsed Time: 0m 21s
[2022-07-16 10:43:18 INFO]  Dev.  Loss: 7.150 | Dev.  Metrics: 56.40% | Elapsed Time: 0m 2s
[2022-07-16 10:43:39 INFO] Epoch: 100 | Step: 2200 | LR: (0.000020/0.001000)
[2022-07-16 10:43:39 INFO]  Train Loss: 1.963 | Train Metrics: 73.62% | Elapsed Time: 0m 21s
[2022-07-16 10:43:42 INFO]  Dev.  Loss: 7.176 | Dev.  Metrics: 56.83% | Elapsed Time: 0m 2s
[2022-07-16 10:43:42 INFO] -------------------------------------------- Evaluating --------------------------------------------
[2022-07-16 10:43:42 INFO] Evaluating on dev-set
[2022-07-16 10:43:44 INFO] ER | Micro Precision: 60.857%
[2022-07-16 10:43:44 INFO] ER | Micro Recall: 54.756%
[2022-07-16 10:43:44 INFO] ER | Micro F1-score: 57.645%
[2022-07-16 10:43:44 INFO] ER | Macro Precision: 41.484%
[2022-07-16 10:43:44 INFO] ER | Macro Recall: 53.931%
[2022-07-16 10:43:44 INFO] ER | Macro F1-score: 42.616%
[2022-07-16 10:43:44 INFO] Evaluating on test-set
[2022-07-16 10:43:46 INFO] ER | Micro Precision: 59.062%
[2022-07-16 10:43:46 INFO] ER | Micro Recall: 45.215%
[2022-07-16 10:43:46 INFO] ER | Micro F1-score: 51.220%
[2022-07-16 10:43:46 INFO] ER | Macro Precision: 55.158%
[2022-07-16 10:43:46 INFO] ER | Macro Recall: 37.900%
[2022-07-16 10:43:46 INFO] ER | Macro F1-score: 44.274%
[2022-07-16 10:43:46 INFO] scripts/entity_recognition.py --dataset WeiboNER --ck_decoder boundary_selection
[2022-07-16 10:43:46 INFO] {'affine_arch': 'FFN',
 'agg_mode': 'max_pooling',
 'batch_size': 64,
 'bert_arch': 'None',
 'bert_drop_rate': 0.2,
 'char_arch': 'None',
 'ck_decoder': 'boundary_selection',
 'ck_size_emb_dim': 25,
 'corrupt_rate': 0.0,
 'dataset': 'WeiboNER',
 'doc_level': False,
 'drop_rate': 0.5,
 'emb_dim': 100,
 'emb_freeze': False,
 'enc_arch': 'LSTM',
 'finetune_lr': 2e-05,
 'fl_gamma': 0.0,
 'grad_clip': 5.0,
 'hard_neg_sampling_rate': 1.0,
 'hard_neg_sampling_size': 5,
 'hid_dim': 200,
 'language': 'Chinese',
 'log_terminal': True,
 'lr': 0.001,
 'max_span_size': 10,
 'neg_sampling_rate': 1.0,
 'num_epochs': 100,
 'num_grad_acc_steps': 1,
 'num_layers': 1,
 'num_neg_chunks': 100,
 'optimizer': 'AdamW',
 'pdb': False,
 'pipeline': False,
 'profile': False,
 'save_preds': False,
 'sb_adj_factor': 1.0,
 'sb_epsilon': 0.0,
 'sb_size': 1,
 'scheduler': 'None',
 'scheme': 'BIOES',
 'seed': 515,
 'sl_epsilon': 0.0,
 'train_with_dev': False,
 'use_amp': False,
 'use_biaffine': True,
 'use_bigram': False,
 'use_crf': True,
 'use_elmo': False,
 'use_flair': False,
 'use_interm1': False,
 'use_interm2': False,
 'use_locked_drop': False,
 'use_softlexicon': False,
 'use_softword': False}
[2022-07-16 10:43:46 INFO] ============================================== Ending ==============================================
syuoni commented 2 years ago

您好，请参考这个链接：https://github.com/syuoni/eznlp/blob/master/docs/boundary-smoothing.md

对中文数据集：

$ python scripts/entity_recognition.py @scripts/options/with_bert.opt \
    --num_epochs 50 \
    --batch_size 48 \
    --dataset {ontonotesv4_zh | SIGHAN2006 | WeiboNER | ResumeNER} \
    --ck_decoder boundary_selection \
    --sb_epsilon {0.0 | 0.1 | 0.2 | 0.3} \
    --sb_size {1 | 2} \
    --bert_arch {BERT_base_wwm | MacBERT_base | MacBERT_large} \
    --use_interm2 \
    [options]
syuoni commented 2 years ago

另外，根据以往 NER 领域多数论文，实验结果报告 Micro F1-score。

Weibo NER 数据量较小，单次实验结果波动较大，建议多次实验后看结果。

lingvisa commented 2 years ago

按照文档中的默认参数，我可以得到 %73.25 的 micro-f1, 试验结果可以重复，还要好点。一个问题是，默认参数的三个项目：

use_crf: True
use_biaffine: True

文章中说使用 biaffine 取代 crf 作为 decoder, 但是为什么默认参数两个都设置为 True? 另外，文章中说 Boundary Smoothing Parameter 是 {0.1, 0.2, 0.3} 之间取值，但运行脚本中默认参数是 ‘0’，这两个参数是一回事吗？ 我试着调整 crf 参数： python scripts/entity_recognition.py --dataset WeiboNER --ck_decoder boundary_selection --no_crf True

但得到错误： entity_recognition.py: error: unrecognized arguments: True

怎么测试指定这个参数？ 谢谢

syuoni commented 2 years ago

您好，请具体参照代码实现来解读参数： （1）如果采用 boundary_selection 作为解码框架，那 use_crf 取值无论是什么都没有影响。CRF 目前仅适配 sequence_tagging 框架。 （2）sb_epsilon 对应论文中 boundary smoothing 的 epsilon。代码里默认值为 0，则不使用 boundary smoothing；需要调整为 0.1/0.2/0.3 等取值以开启。 （3）如果在 sequence_tagging 框架下，可以用 --no_crf 禁用 CRF （不需要后面跟 True）。

如果只看变量名，容易望文生义，请结合代码和注释来解读，谢谢！

lingvisa commented 2 years ago

好的。如果使用自己的数据，关于自己准备的数据格式你有什么建议吗？因为不同公共数据集的格式不一样。另外，如果多 GPU训练，有没有快速设置可以达到使用多个 GPU? 谢谢

On Mon, Jul 18, 2022, 6:49 PM Enwei Zhu @.***> wrote:

您好，请具体参照代码实现来解读参数： （1）如果采用 boundary_selection 作为解码框架，那 use_crf 取值无论是什么都没有影响。CRF 目前仅适配 sequence_tagging 框架。 （2）sb_epsilon 对应论文中 boundary smoothing 的 epsilon。代码里默认值为 0，则不使用 boundary smoothing；需要调整为 0.1/0.2/0.3 等取值以开启。 （3）如果在 sequence_tagging 框架下，可以用 --no_crf 禁用 CRF （不需要后面跟 True）。

如果只看变量名，容易望文生义，请结合代码和注释来解读，谢谢！

— Reply to this email directly, view it on GitHub https://github.com/syuoni/eznlp/issues/29#issuecomment-1188505944, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4J634NEIGNZPPDACHQZHDVUYCRJANCNFSM53XBXM6A . You are receiving this because you authored the thread.Message ID: @.***>

syuoni commented 2 years ago

数据格式推荐用 json 吧，比较灵活通用；形式类似 https://github.com/syuoni/eznlp/blob/master/data/conll2004/ 下的几个样例文件。

本仓库目前没有支持多 GPU 的设置。如果想要实现，可以参考：https://github.com/tczhangzhi/pytorch-distributed

lingvisa commented 2 years ago

好的，感谢。