particle-based variational inference

[EricWang12]

Hello,

Thank you so much for your great work, it helps a lot in understanding the papers and concepts.

However, I am very much confused about the particle-based variational inference in Prolific Dreamer and hope to get some insights from your code, but I could not find any reference. Not sure if I understand it correctly, but it seems theta is represented by the "latent" at line227 ? . Does that mean particle-based variational inference is not included in the code piece?

Thanks again for the great work and I look forward to your reply :)

[yuanzhi-zhu]

The latent you mentioned is the theta representation of particles, where the batch size should be the number of particles.

According to my understanding, there should be an overall batch-size equal to the number of particles, a sub-batch-size (batch size of U-Net) for selecting sub-batch particles to train with VSD loss, and another sub-batch-size (batch size of particles) for selecting sub-batch particles to train the LoRA or simple UNet. Currently, there is only one variable called batch size in the code. Namely, these three batch-sizes are set to be the same for demonstration purposes.

Hope I got it correct and this answers your question ;)

[yuanzhi-zhu]

Had an updation, and this is what I got for 16 particles

[EricWang12]

Interesting! That helps a lot, thanks so much!! :)

[cwchenwang]

@yuanzhi-zhu I have a question regarding the number of particles? Why the implementation still works quite well with particles = 1, but according to the paper, we should have multiple particles.

[yuanzhi-zhu]

@cwchenwang the paper also claims that particle == 1 still works

[cwchenwang]

Thank you. But that's a bit different from their theory.

[yuanzhi-zhu]

I understand your concerns.

To my comprehension, the original paper solely derives the dynamic equation for particles. Under adequate assumptions, each particle would move toward the target distribution.

However, we still need a phi model to represent a delta distribution of the particles when the number of particles is 1, which to me is quite intricate, too...

I hope this clarifies my understanding, and you may email the authors for further explanation.