cemoody / lda2vec

MIT License
3.15k stars 628 forks source link

factor initialization #81

Open bhomass opened 6 years ago

bhomass commented 6 years ago

Any one see the logic in how the factors were initialized? apparently using QR decomposition to obtain 2 orthogonal matrices, dot product them by the lower of the two matrix dimensions, then divide by the square root of (n_topics + n_dim)

Does this guarantee a simplex condition? what exactly does this calculation do for you?

bhomass commented 6 years ago

found this

https://stackoverflow.com/questions/38426349/how-to-create-random-orthonormal-matrix-in-python-numpy

bhomass commented 6 years ago

Having determined that the initialization aims to generate orthonormal random vectors, I see no basis in that requirement at all from the original paper. What would you want to impose orthogonality among the initial topic mixture?