search

microsoft / LightLDA

Scalable, fast, and lightweight system for large-scale topic modeling
http://www.dmtk.io
MIT License
842 stars 235 forks source link

Fatal in alias index: word 46631 not exist #34

Open liuxialong opened 8 years ago

liuxialong commented 8 years ago

[INFO] [2016-06-16 04:22:00] Actual Model capacity: 186 MB, Alias capacity: 301 MB, Delta capacity: 256MB [INFO] [2016-06-16 04:22:00] INFO: block = 0, the number of slice = 2 [INFO] [2016-06-16 04:22:00] Server 0 starts: num_workers=1 endpoint=inproc://server [INFO] [2016-06-16 04:22:00] Server 0: Worker registratrion completed: workers=1 trainers=8 servers=1 [INFO] [2016-06-16 04:22:00] Rank 0/1: Multiverso initialized successfully. [INFO] [2016-06-16 04:22:01] Rank 0/1: Begin of configuration and initialization. [INFO] [2016-06-16 04:22:26] Rank 0/1: End of configration and initialization. [INFO] [2016-06-16 04:22:26] Rank 0/1: Begin of training. [DEBUG] [2016-06-16 04:22:26] Request params. start = 1, end = 162931 [INFO] [2016-06-16 04:22:27] Rank = 0, Iter = 0, Block = 0, Slice = 0 [DEBUG] [2016-06-16 04:22:27] Request params. start = 162932, end = 1099301 [INFO] [2016-06-16 04:22:27] Rank = 0, Alias Time used: 1.77 s [INFO] [2016-06-16 04:23:01] Rank = 0, Training Time used: 270.57 s [INFO] [2016-06-16 04:23:01] Rank = 0, sampling throughput: 21328.182632 (tokens/thread/sec) [INFO] [2016-06-16 04:23:08] doc likelihood : -8.291677e+08 [INFO] [2016-06-16 04:23:08] word likelihood : 2.141731e+09 [INFO] [2016-06-16 04:23:08] Normalized likelihood : -7.755637e+09 [INFO] [2016-06-16 04:23:08] Rank = 0, Evaluation Time used: 59.11 s [INFO] [2016-06-16 04:23:08] Rank = 0, Iter = 0, Block = 0, Slice = 1 [FATAL] [2016-06-16 04:23:09] Fatal in alias index: word 447 not exist [FATAL] [2016-06-16 04:23:09] [FATAL] [2016-06-16 04:23:09] [FATAL] [2016-06-16 04:23:09] [FATAL] [2016-06-16 04:23:09] Fatal in alias index: word 429 not exist Invalid topic assignment 934165176 from word proposal [FATAL] [2016-06-16 04:23:09] Fatal in alias index: word 1202 not exist Fatal in alias index: word 2893 not exist [FATAL] [2016-06-16 04:23:09] Invalid topic assignment 412339115 from word proposal [FATAL] [2016-06-16 04:23:09] Fatal in alias index: word 46631 not exist Fatal in alias index: word 2946 not exist [FATAL] [2016-06-16 04:23:09] Fatal in alias index: word 46631 not exist [FATAL] [2016-06-16 04:23:09] Fatal in alias index: word 429 not exist

liuxialong commented 8 years ago

Can't find alias index when training Slice1.

feiga commented 8 years ago

This should be the same problem with issue https://github.com/Microsoft/lightlda/issues/15