about the function of reestimation

[tzm-tora]

hi, thank you for opensourcing your code. I read about the part of VQGAN in your code and I do not understand the function of reestimation. I also found it seems that this operation is slow during the training. Can you briefly introduce me the function of reestimation? Thank you.

[guyuchao]

At the beginning of codebook training, the codebook usage is low. So we need to use offline KMeans to re-initialize its codebook items. If you want to speed up this function, you may set (n_init=1) in the KMeans algorithm. And the origin setting in sklearn is n_init=10, which means the KMeans will repeat 10 times.

cluster.k_means(encodings, num_clusters, n_init=1)

[tzm-tora]

Thank you for your explaination. I am new to VQGAN, and I am curious about if this trick is a common operation for codebook training or this is first proposed by your team. Thank you!

[guyuchao]

It is not our contribution, you can refer to "robust training of vector quantized bottleneck models" for details.