search

kmkurn / pytorch-crf

(Linear-chain) Conditional random field in PyTorch.
https://pytorch-crf.readthedocs.io
MIT License
943 stars 152 forks source link

Unchecked Initialize params in Init #1

Closed farizikhwantri closed 6 years ago

farizikhwantri commented 6 years ago

When training torchcrf module version (0.3.1) using pytorch version 0.2.4, I encounter NaN loss from forward computation. possibly because CRF parameters not initialized before

Here is the log when passing one instance from lstm module 300 dims to nn.Linear then to torchcrf module

lstm hidden :
2017-12-29 09:07:17,238 - INFO - root - Variable containing:
( 0 ,.,.) = 

Columns 0 to 8 
  -0.3024  0.0548 -0.0000  0.0135  0.1097 -0.1398 -0.0569  0.1195 -0.0071
 -0.3441  0.2693 -0.0000 -0.1563 -0.0391  0.1706  0.0360 -0.1091  0.3121
 -0.3552  0.0395  0.0000 -0.1029  0.4409 -0.1020  0.2578 -0.2286 -0.0193

Columns 9 to 17 
  -0.1107 -0.0007 -0.1530 -0.0000 -0.1692 -0.0962  0.1139  0.0132  0.1538
  0.0558  0.0195  0.0275 -0.0000  0.0698  0.2186  0.2312  0.1365 -0.0382
  0.3324  0.1442 -0.2000 -0.0000 -0.1615  0.0404 -0.0260  0.2240 -0.1211

Columns 18 to 26 
  -0.1879 -0.0071 -0.0625  0.3014 -0.1089  0.1881 -0.3226 -0.2328  0.1622
 -0.2262 -0.2449 -0.2407 -0.0109 -0.0068  0.3066 -0.4911 -0.0048  0.0169
  0.0021 -0.0269 -0.0051  0.1964  0.0030 -0.0820 -0.2229 -0.0990  0.0310

Columns 27 to 35 
  -0.1022 -0.1103  0.0386  0.1079  0.1450 -0.0188 -0.0616  0.0000  0.1597
  0.1801  0.1645  0.0948  0.1418  0.2858 -0.3416 -0.2020 -0.0000  0.0428
  0.0588  0.3452 -0.0155  0.0919 -0.2397 -0.1481 -0.1706 -0.0000 -0.1262

Columns 36 to 44 
  -0.1714  0.0000  0.0649  0.1468  0.0170 -0.3933  0.0553 -0.0761  0.4328
 -0.3755  0.0000  0.2756  0.1469 -0.2402 -0.0848  0.0758 -0.0185  0.1007
 -0.0883  0.0000 -0.1557  0.1176  0.1104 -0.1737  0.0776 -0.0709  0.0133

Columns 45 to 53 
  -0.1138 -0.0721 -0.2555 -0.1136  0.0650 -0.2853 -0.0383 -0.1924 -0.0878
 -0.0104  0.0393 -0.2041  0.1408  0.0427 -0.0282  0.0322 -0.3089  0.2671
  0.1139  0.1691 -0.1021 -0.3851  0.1300 -0.1518 -0.2397 -0.1741  0.1581

Columns 54 to 62 
  -0.0363 -0.0600 -0.1095 -0.0233 -0.0011  0.0741 -0.1148  0.0705 -0.0301
  0.0204 -0.0919 -0.1292  0.0038 -0.3615  0.2765 -0.0031 -0.3507 -0.0910
 -0.5493  0.0463 -0.1996 -0.0512  0.1703  0.0860  0.0686  0.1487  0.1957

Columns 63 to 71 
  -0.0441 -0.0000  0.1158  0.0000 -0.1326 -0.2144 -0.3385 -0.2347 -0.0383
 -0.1015 -0.0000 -0.2969 -0.0000  0.0072 -0.0106 -0.2851 -0.0488 -0.0822
  0.1214 -0.0000  0.0413  0.0000 -0.0001  0.0167 -0.0139 -0.1047 -0.0495

Columns 72 to 80 
   0.0000  0.0686 -0.0375 -0.2266  0.0000  0.0605  0.2789  0.0000 -0.1027
  0.0000 -0.0458 -0.0128  0.3314  0.0000 -0.0077  0.1982 -0.0000 -0.2339
  0.0000 -0.1153  0.1715 -0.2791  0.0000  0.2872 -0.0257  0.0000  0.1475

Columns 81 to 89 
  -0.0693  0.1278 -0.2081  0.1067  0.0601 -0.2345  0.0000 -0.1261  0.0998
 -0.1327 -0.0881 -0.0285  0.0020  0.1344 -0.3387  0.0000 -0.2330  0.4240
 -0.1269  0.0222 -0.0881  0.0822  0.0633 -0.0394  0.0000 -0.0447  0.1005

Columns 90 to 98 
   0.0601 -0.2818 -0.0915 -0.0000 -0.1667  0.0736  0.0502  0.0000 -0.2093
  0.0734 -0.3057  0.0352 -0.0000 -0.2095 -0.2647  0.4264  0.0000  0.1107
 -0.1155  0.1542 -0.1582 -0.0000 -0.0560  0.0783 -0.1361  0.0000 -0.0103

Columns 99 to 107 
  -0.1206  0.0846 -0.0680  0.0378 -0.1582 -0.0099  0.1485  0.0000  0.0000
  0.1400 -0.0458 -0.0598  0.1909 -0.3910 -0.6758 -0.2302 -0.0000  0.0000
 -0.2916  0.0088  0.0951 -0.1761 -0.0085 -0.1792 -0.2064 -0.0000  0.0000

Columns 108 to 116 
  -0.1337  0.2123 -0.1010 -0.1257  0.0000 -0.1044  0.1301 -0.3993 -0.1017
 -0.1513  0.1115  0.1221 -0.1946 -0.0000 -0.0130 -0.0254 -0.0542 -0.0974
 -0.2906  0.0156  0.2102  0.0105  0.0000 -0.1112 -0.0140  0.0742  0.0151

Columns 117 to 125 
  -0.0189  0.1031  0.2555 -0.0443  0.0766  0.0084 -0.0230  0.0501 -0.3675
  0.2021 -0.2891 -0.0351 -0.1743  0.3122  0.0760 -0.0478 -0.3143 -0.2035
  0.2177  0.2749  0.0076  0.0403  0.1036 -0.0337 -0.1401 -0.0082  0.0270

Columns 126 to 134 
  -0.0578 -0.0108 -0.1187  0.1386 -0.0156 -0.0056  0.0103 -0.0659  0.2106
  0.0096  0.0461 -0.1115  0.3557  0.1699 -0.0051 -0.1683 -0.0401 -0.0644
 -0.0448  0.0751  0.1242  0.0328 -0.2548 -0.0715  0.0221  0.0969 -0.0621

Columns 135 to 143 
   0.1960 -0.0476  0.2171  0.1671  0.1393 -0.0252 -0.0192 -0.0656 -0.2273
  0.0980 -0.0930  0.2376  0.1949 -0.1071  0.1856 -0.1145 -0.0927  0.3215
 -0.2770 -0.2060  0.1390  0.1908  0.1614  0.1142 -0.0390 -0.1477 -0.1599

Columns 144 to 152 
  -0.0614 -0.3834 -0.1685  0.3919 -0.2907 -0.0786  0.0313  0.0743 -0.0000
 -0.1174 -0.0375 -0.2633  0.3274 -0.2755 -0.2314  0.0491  0.1064 -0.0000
  0.0198  0.0110 -0.1439  0.1322 -0.1293 -0.1125  0.1284  0.0114  0.0000

Columns 153 to 161 
   0.0000 -0.0678  0.0117 -0.0224  0.0984  0.0000  0.3218 -0.1375  0.0000
 -0.0000  0.0583 -0.0360 -0.8546  0.0323  0.0000  0.3010  0.0919 -0.0000
 -0.0000 -0.0774  0.1754 -0.0606  0.1098 -0.0000 -0.0122 -0.1889  0.0000

Columns 162 to 170 
  -0.0686  0.0000 -0.0697  0.0901 -0.0578 -0.0486 -0.2314 -0.2631 -0.3633
  0.0288 -0.0000 -0.1322 -0.0271  0.2687 -0.2016 -0.2042 -0.2741 -0.2876
  0.0897  0.0000 -0.0039  0.0093 -0.1374 -0.1038  0.0054 -0.1721  0.2671

Columns 171 to 179 
   0.0728  0.0245  0.2898  0.0913  0.3150 -0.0550  0.1246 -0.1551  0.0648
 -0.0429  0.3847  0.0912  0.0331  0.3161 -0.0667  0.3400 -0.0388 -0.0036
  0.0877  0.1819  0.1225  0.0621 -0.0355 -0.2585  0.0713 -0.0841  0.2578

Columns 180 to 188 
  -0.1895  0.1002 -0.0052  0.3516 -0.1101  0.0523 -0.0000  0.2091  0.0747
 -0.1607 -0.0046 -0.1061 -0.0385 -0.1413 -0.0171  0.0000  0.1063 -0.4841
  0.1050  0.1690  0.1147  0.2506  0.2040  0.0545  0.0000  0.2060 -0.0623

Columns 189 to 197 
   0.0600 -0.0050 -0.0401 -0.0000 -0.1340  0.1341 -0.0451  0.1836 -0.0808
 -0.0280 -0.3557 -0.1866 -0.0000 -0.3596  0.1937 -0.0347  0.1154  0.0015
  0.1930  0.1141 -0.2795 -0.0000 -0.0629  0.3187  0.1487  0.0753 -0.0735

Columns 198 to 206 
   0.2044 -0.2763  0.0000  0.2526  0.0407 -0.1678 -0.0715  0.2023 -0.0038
  0.0858  0.0360 -0.0000  0.0356 -0.0155 -0.0775  0.3066  0.1852 -0.0844
  0.1613 -0.0127 -0.0000  0.0738 -0.1326 -0.0106  0.1497  0.6372  0.0351

Columns 207 to 215 
   0.0383  0.1690 -0.0000 -0.1033 -0.1101  0.0000 -0.0876  0.0957 -0.0977
  0.1350  0.0972 -0.0000  0.0241 -0.3232  0.0000 -0.0625 -0.2341 -0.0584
  0.0540  0.0776 -0.0000  0.1214  0.1804  0.0000 -0.0839  0.4604  0.2396

Columns 216 to 224 
   0.0508  0.0994 -0.1016 -0.0899 -0.1522  0.0000  0.1327  0.1221 -0.1373
  0.1690 -0.0343  0.0547  0.2222 -0.0506 -0.0000  0.0177  0.0349 -0.1143
  0.1727  0.1096  0.0626  0.0436  0.1396  0.0000  0.0528 -0.0249  0.2179

Columns 225 to 233 
  -0.0845 -0.1163  0.0705 -0.1589 -0.0378 -0.1994  0.2543  0.1034  0.2005
  0.0094 -0.0755  0.0915 -0.0504  0.0498 -0.0067  0.0800  0.1875  0.1883
  0.0364 -0.0549  0.0915  0.2024  0.1746  0.0507  0.1951  0.2767  0.0016

Columns 234 to 242 
  -0.0362  0.0638  0.0865  0.0119 -0.0331 -0.0068  0.0037  0.0220  0.0385
  0.0448  0.0941  0.2425 -0.2410  0.0814 -0.3031 -0.0512 -0.2177  0.0450
 -0.0252  0.3924  0.0181 -0.0210  0.0322 -0.1107  0.0121  0.2406 -0.0365

Columns 243 to 251 
  -0.0096 -0.0000  0.1289 -0.0000  0.0000  0.1612 -0.0654 -0.2008  0.0145
  0.1697  0.0000  0.1053 -0.0000  0.0000  0.1589  0.2171  0.1363 -0.0249
  0.1692 -0.0000  0.1759 -0.0000 -0.0000 -0.3958  0.0598 -0.1577 -0.1894

Columns 252 to 260 
  -0.1549  0.3160  0.0221  0.0000  0.1242  0.1123  0.1337 -0.0214 -0.0000
 -0.2198  0.1118 -0.2462 -0.0000 -0.2487  0.2292  0.0541  0.1150 -0.0000
  0.3288  0.2174  0.0088  0.0000 -0.0211 -0.0540 -0.0332 -0.0510  0.0000

Columns 261 to 269 
   0.1102 -0.0370  0.2161  0.2082  0.2646 -0.1688  0.0222  0.0684  0.0252
  0.1097 -0.0793  0.0189 -0.2266  0.0223  0.1617 -0.0686  0.3449  0.0602
 -0.0252 -0.1221  0.1081  0.2685  0.0149 -0.1512 -0.1749 -0.0512  0.0889

Columns 270 to 278 
  -0.0000  0.1140 -0.1641  0.0528 -0.1709  0.0775 -0.1427 -0.0161 -0.2229
 -0.0000 -0.1343 -0.1599 -0.1046  0.0678  0.2201 -0.2377 -0.1704 -0.3747
  0.0000  0.0893 -0.0924  0.2425  0.1202  0.1105 -0.1062 -0.1071 -0.1847

Columns 279 to 287 
  -0.0000 -0.1083 -0.0000 -0.0220  0.0487 -0.3342 -0.1988  0.0566 -0.0731
  0.0000  0.4410 -0.0000  0.2042  0.0569  0.0335 -0.1195  0.1861 -0.0210
 -0.0000  0.2180 -0.0000  0.1403  0.0247  0.0461 -0.1210 -0.0353  0.0776

Columns 288 to 296 
   0.2324  0.0000 -0.1793  0.3275  0.0456  0.0506  0.2650  0.0422 -0.0203
  0.1437 -0.0000 -0.0049  0.1338 -0.2079  0.0761  0.5466 -0.2304 -0.2335
  0.1686  0.0000  0.1794  0.1843 -0.0113 -0.1217  0.1163  0.5834 -0.3259

Columns 297 to 299 
  -0.2776  0.0000 -0.0008
 -0.2349  0.0000  0.0936
  0.1223 -0.0000  0.0938
[torch.FloatTensor of size 1x3x300]

logits / emission : 
2017-12-29 09:07:17,242 - INFO - root - Variable containing:
(0 ,.,.) = 

Columns 0 to 8 
  -0.0727 -0.1339 -0.0356  0.0746  0.0122 -0.0317 -0.0317  0.0196 -0.0424
 -0.1730 -0.0573  0.1282 -0.0520 -0.2719 -0.1338  0.1333  0.0369 -0.0064
 -0.0886 -0.0453  0.0726 -0.2096  0.1002 -0.1149  0.0496  0.1408  0.1913

Columns 9 to 12 
   0.0669  0.0864  0.0406  0.0853
  0.1540  0.0856 -0.1543  0.2038
  0.0946 -0.0216 -0.0473  0.1621
[torch.FloatTensor of size 1x3x13]

loss : 
2017-12-29 09:07:17,244 - INFO - root - Variable containing:
nan
[torch.FloatTensor of size 1]