m-tassano / fastdvdnet

FastDVDnet: A Very Fast Deep Video Denoising algorithm
MIT License
586 stars 125 forks source link

Choice of cascading level and number of frames per level #29

Closed XiangY-Q closed 3 years ago

XiangY-Q commented 3 years ago

Hi, thanks for your work and the pre-trained model. Two questions not really related to the repo itself.

  1. Regarding architecture, have you experimented with more cascading levels/steps (say, 3-steps denoising so information from a longer sequence might be exploited) or more number of frames for each denoising block (also lead to usage of longer sequence)? Do you observe/expect any improvement by leveraging more frames?

  2. For image denoisers, a mismatch between input noisy image and noise map (say, actual input noise level 25 but input noise map 50) could lead to catastrophic failure of denoising networks. Do you observe similar phenomenon? What's the target noise range of the pretrained model provided?

Thanks:P

m-tassano commented 3 years ago

Hi, Regarding 1) I suggest you reading the supplementary material where I included some ablation studies. Basically, there are no significant changes in performance when using more input frames.

As for 2), the model was trained with sigmas in [5, 55]. The mismatch example you give is a severe one. In particular, I would expect the model to largely overdenoise the input, as any other denoiser would do. However, this is only what I expect would happen in this scenario, I haven't experimented much in this regard.

Hope this helps