Question about input to optical flow estimator

NVlabs / PWC-Net

PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume, CVPR 2018 (Oral)

Other

1.64k stars 355 forks source link

Question about input to optical flow estimator #46

Closed yzcjtr closed 6 years ago

yzcjtr commented 6 years ago

Hi, I have a question about the input to the flow estimator. As mentioned in the paper, the input is:

the cost volume, features of the first image, and upsampled optical flow

Then I notice in the code that the input also consists of upsampled features from previous scale, such as up_feat6 in x = torch.cat((corr5, c15, up_flow6, up_feat6), 1). Could you please give some insights on incorporating this kind of information? Thanks.

deqings commented 6 years ago

That's a very good question. Empirically we found that adding the upsampled features slightly improves the performance (see this related issue https://github.com/NVlabs/PWC-Net/issues/33). Intuitively, the upsampled features may provide some global contextual information, complementary to the upsampled optical flow.