search

mattiasxu / Video-VQVAE

VQVAE for video prediction
MIT License
26 stars 7 forks source link

Seperate training for bottom and prior model #3

Closed dongkyunk closed 2 years ago

dongkyunk commented 2 years ago

I've been working on this model, and the pixel snail training code didn't converge.

I found out that the problem was that the original code was training the top and bottom prior models simultaneously, whereas the paper trained the two models in seperate.

Changing the order of the top and bottom generation can ensure that the gradients of the two models flow seperately and helps the model converge.

mattiasxu commented 2 years ago

Thanks!