search

SensorsINI / v2e

V2E: From video frames to DVS events
https://sites.google.com/view/video2events/home
MIT License
290 stars 50 forks source link

Slomo frame insertion problem #54

Open yanchi-3dv opened 1 year ago

yanchi-3dv commented 1 year ago

Thanks for your nice work！ I try to use your code to generate events from video, however, frame insertion results show like below.

https://github.com/SensorsINI/v2e/assets/96414864/eb786b4f-64b5-4c9e-b7a8-c45baaca940d

In the process of acceleration motion of the camera, the video obtained by inserting frames will have serious blurring. I wonder if the problem is caused by the optical flow insertion frame or my parameters are not set properly. And how can I improve the performance. Thanks!

tobidelbruck commented 1 year ago

Could you please share the source video? I suspect the frame shifts are too large or else SuperSloMo just does not generalize to this data. It's probably time to update the synthetic slomo model but it could be a lot of work. SuperSloMo estimates interframes between the last 2 adjacent frames using estimated optical flow. Any newer/better model with the same form could be used. And SuperSloMo could be fine tuned further past the monocrhomme model that we trained for v2e.