Motion Vectors and Residuals for Salience Detection

[manza-ari]

Respected @chaoyuaw and @manzilzaheer

I am thinking to use motion vectors (extracted from compressed videos) to find the attention point of salience detection for Human Action Recognition. What is your opinion about this?

[chaoyuaw]

I'd guess that MVs likely contain useful information for saliency detection. However, it's also noisier and in lower resolution than RGB frames. Maybe using both MVs and RGB frames can be something interesting to try.

Maybe some of the techniques discussed in https://arxiv.org/pdf/1803.07742.pdf might be relevant, as it also performs pixel-level task with compressed videos.

[manza-ari]

Thank you so much for your reply. That really means alot to me.