search

guanyingc / DeepHDRVideo

HDR Video Reconstruction: A Coarse-to-fine Network and A Real-world Benchmark Dataset (ICCV 2021)
https://guanyingc.github.io/DeepHDRVideo
Other
98 stars 19 forks source link

Result on test sequence Carousel_fireworks #18

Closed yorunouta closed 1 year ago

yorunouta commented 2 years ago

Hi,

I notice in one issue, you think vimeo-90k dataset is not perfect but effective in getting the higher performance. So now I'm preparing training with/without it to see the difference and considering how to make another suitable synthetic dataset.

However, after training without vimeo-90k, I found results on test sequence Carousel_fireworks are obviously lower than other sequences. PSNR on Carousel_fireworks is 31.62, while PSNR on Poker_fullshot is 41.31 (2-alternating exposed inputs). Although training with vimeo-90k is not finished, I think maybe PSNR on Carousel_fireworks is still bad.

So do you also get a bad PSNR score on Carousel_fireworks in your work? And I also wonder about the reason.

guanyingc commented 2 years ago

Hi, if you check the data in Carousel_fireworks, you will find that carousel firework dataset consists of bright lights with large motion in a dark scene, which is far more challenging than the poker_fullshot.

yorunouta commented 2 years ago

Yes, the scene in carousel firework is more complex. If it's convenient, could you please tell me the results on carousel fireworks and poker fullshot in your experiment? (you showed avarage scores in the paper)

guanyingc commented 2 years ago

I don't remember now. But you can donwload the precomputed results in Results/ at Google Drive and you can compute the results by yourself.

The command will compute the average score on two sequences. https://github.com/guanyingc/DeepHDRVideo/blob/master/scripts/test_synthetic_2exp.sh

guanyingc commented 2 years ago

Hi, could you report your results of model trained with both synthetic datasets?

yorunouta commented 2 years ago

Thanks for the reply.

Now training with both datasets is not finished, so I only show results about training with synthetic training dataset. For poker_fullshot: PSNR-T / HDR-VDP-2 / HDR-VQM 41.31 73.2 84.38

For carousel_fireworks: PSNR-T / HDR-VDP-2 / HDR-VQM 31.62 67.07 71.25

And I also make 2 more evaluation sequences with the help of your dataset code repository. The results are pretty good. For poker_fullshot: PSNR-T / HDR-VDP-2 / HDR-VQM 46.38 76.87 91.28

For fishing_longshot: PSNR-T / HDR-VDP-2 / HDR-VQM 41.98 72.78 90.61

It seems that only results of carousel_fireworks are inexpectant. After finishing overall training, I will evaluate again to compare.