Performance on Synthetic Driving Dataset?

[SwagJ]

Hi @sniklaus ,

Thank you for your great work. I am just wondering if you have tested PWC's performance on synthetic driving dataset(https://lmb.informatik.uni-freiburg.de/resources/datasets/SceneFlowDatasets.en.html). In theory, PWC's performance should be good on this dataset, given that initial training of PWC is on FlyingThings3D. However, on this dataset, performance on syntheic driving is about 28, which is very large. Do you have any clue?

I am looking forward to your reply. Thank you very much.

Best,

[sniklaus]

I am afraid that I have not conducted any experiments on the synthetic driving dataset you linked. I agree with you that, based on intuition, one would expect a network trained on FlyingThings3D to do reasonably well on a synthetic driving dataset. However, the types of motion in FlyingThings3D are probably quite different from a camera mounted to a (virtual) car driving through a city. So intuition goes both ways. I am afraid that I do not have any recommendation on how to improve the poor performance that you are experiencing. Note that this repository is a reimplementation of PWC-Net and you could try to reach out to the PWC-Net authors (Deqing et al.) as well to see what they have to say. Alternatively you could also give another method a try (but your mileage may vary): https://github.com/princeton-vl/RAFT