search

aakhundov / tf-example-models

TensorFlow-based implementation of (Gaussian) Mixture Model and some other examples.
Apache License 2.0
43 stars 13 forks source link

Question on the result #2

Closed junhyeokahn closed 6 years ago

junhyeokahn commented 6 years ago

How do you actually get a result shown in gmm-best.png. I mean, among several algorithms, which one did you use to get the best result. By just running your codes (e.g. tf_gmm_diag.py, tf_gmm_full.py, tf_gmm_grad, tf_gmm.py), they mostly give incomplete result as below.

screen shot 2018-03-23 at 9 28 35 am

Is gmm sensitive to initial mean assignment?

aakhundov commented 6 years ago

Hello,

Sorry for the late answer, I somehow missed your question.

GMM (as well as K-Means) is indeed quite sensitive to initial mean assignments. So, if the initial mean values don't cover all the clusters in the data, t's quite hard for the algorithm to properly reassign them. What people frequently do in practice to circumvent this is finding the initial mean assignment using cheaper K-Means, and then running an EM to train a GMM (see Bishop's "PRML", page 438).

The results shown in the "gmm-best.png" (as well as "kmeans-best.png") are the best results obtained from a series of tf_gmm.py (and tf_kmeans.py) runs (which is kind of manifested in the file names :)).

Hope this helps!