search

uzh-rpg / rpg_timelens

Repository relating to the CVPR21 paper TimeLens: Event-based Video Frame Interpolation
599 stars 86 forks source link

Different results between paper and code test #32

Open EMJian opened 2 years ago

EMJian commented 2 years ago

Hey, It's a pretty work. I also conduct some comparison experiments in video frame interpolation. But I have several questions in some details. I've been trying to run this code and to output results in timelens. However, in terms of PSNR and SSIM, values of metrics from timelens code are different that from your paper 'Time Lens'. I don't know what's wrong.

Some details are provided as follow, Dataset: HSERGB, BS-ERGB Test code: uzh-rpg/rpg_timelens Evaluation code: rpg_event_based_frame_interpolation_evaluation

Like your paper 'Time Lens', I report PSNR and SSIM for all sequences by skipping 1, 5 and 7 frames respectively, and reconstructing the missing frames.

My results (mean+std): 1) skip 5 frames:

PSNR/SSIM HSERGB(far) HSERGB(close)
code(timelens) 31.33±2.55/0.883±0.069 31.81±4.20/0.822±0.108
paper(timelens) 33.13±2.10/0.877±0.092 32.19±4.19/0.839±0.090

2) skip 7 frames:

PSNR/SSIM HSERGB(far) HSERGB(close)
code(timelens) 30.05±2.24/0.864±0.065 31.54±6.05/0.844±0.120
paper(timelens) 32.31±2.27/0.869±0.110 31.68±4.18/0.835±0.091

3) skip 1 frame:

PSNR/SSIM BS-ERGB
code(timelens) 24.03±4.30/0.741±0.153
paper(timelens) 28.56/-

In addition, I noticed and solved this issue for HSERGB dataset, as you mentioned before. """ In events_aligned each event file with index n contains events between images with index n-1 and n, i.e. event file 000001.npz contains events between images 000000.png and 000001.png. """ So, I deleted event file '000000.npz' of each sequence to make sure that all steps are correct for HSERGB dataset. BS-ERGB dataset are still unchanged. Other additional things are not done. If I have any mistakes please correct me. What confuses me is that the results obtained using the code are different from the results of the paper. Looking forward to your reply.

wefvdge commented 1 year ago

Hi!I have some questions about BS-ERGB dataset.I found the x, y in raw event data is much greater than the resolution of RGB image. So do you find out the correspondence of spatial coordinates between event data and image?thanks a lot!

rayray9999 commented 3 months ago

Hello, I test BS-ERGB and get timelens: PSNR: mean: 24.02892219635027 std: 4.295050466411185 SSIM: mean: 0.7409115504636926 std: 0.152825300806362

which is almost same as your result, did you figure out it was due to some wrong setting or it's really a mismatch between paper and test?