cuijl-ai
commented
2 months ago In mini-batch OT-CFM, I noticed that z is a pair sampled from $\pi(x_0, x_1)$. So I think in this paper, it is also a pair of $(x_0, x_1)$. Because the (image, depth) pair already satisfies the OT map, so the authers used $(x_0, x_1)$ directly to train a vector field, instead of computing and resampling a OT strategy $\pi(\cdot, \cdot)$.
The question is what was used as $x_0$ when training, the original image (encoded in latent space), or the noised version of it?
Hi!
Congrats for the great work.
I have a question regarding the flow matching formulation in the paper. In many parts of the paper you refer to X_0 as the clean image and X_1 as its depth map, aiming to learn the flow between the two distributions. But the training pipeline figure seem to depict that you predict the flow for a noised version of the real image, rather than of the interpolation of the image latent and the depth latent, as stated in equation (4) (and as aligns with the flow matching formulation, to my understanding). Additionally, in both eq (2) and (4), you also have z in the LHS of the equation, but not on the RHS, so I wasn't sure what was z there.
Could you please clarify your training paradigm? Essentially my question is -- do you define x_t as an interpolation of the depth and the image, or of the image and random noise? If it's the latter- Could you please explain how it aligns with the flow matching formulation, and with eq (4) in the paper?
Thanks in advance,
Michal