A question about motion vector

[xizaoqu]

Hi, I visualize the extracted motion vector by mv-extractor, it seems kind of noisy, will it influence the training and validation performance of video generation?

https://github.com/jy0205/LaVIT/assets/45515569/afd99d47-d9dc-4581-9ebb-1b7cc9225803

[jy0205]

Compared to dense optical flow, the motion vector is a sparser representation (motion representation of 16x16 pixel macro-blocks, rather than each pixel, so may be kind of noisy); According to our experimental conclusions, using motion vector is enough to serve as a strong condition to reconstruct the original video.

[xizaoqu]

Thanks for your reply.