search

JiangtaoNie / UAL

Unsupervised Adaptation Learning for Hyperspectral Imagery Super-resolution
33 stars 11 forks source link

Question on used kernels and other details in training the fuse module #1

Open JingyunLiang opened 4 years ago

JingyunLiang commented 4 years ago

As shown in the below line https://github.com/JiangtaoNie/UAL/blob/5793e197ca05e7b989a650cf095e87c8a76a9940/Train_FusionModel.py#L38 It seems that only five different Gaussian kernels are used in training the fuse modules, but they are shown to generalize well to kernels in Fig. 5 of the paper.

1, How did you choose these five kernels? 2, Did you try other sampling, e.g. uniform sampling ? 3, Does it generalize to other arbitrary Gaussian /non-Gaussian kernels? 4, Are the estimated kernels in the degeneralization module similar to ground-truth ones? 5, The spectral downsampler is fixed. Does it also act as an extra regularization in Eq. 4 of the paper? 6, MHF-net and other works are not trained with noisy LR images. Did you try to train a noisy version for comparison?

Thank you.

JiangtaoNie commented 4 years ago

Yes, this work is mainly to force the trained network to adapt to unseen spatial degenerations. Specificly, we utilize five different settings Gaussian kernel to generate the LR HSI for training and adopt four motion kernels to generate the test LR HSI. In supplementation of the main paper can find the estimated kernel.
Reply to exact quetions:

  1. The five kernels is choosen randomly by adjusting the kernel size and variance of Gaussian kernel just for inproving the diversity.
  2. Not yet, we will try it in subsequent work.
  3. So far, we have tested to generalize the model from Gaussian to kinds of motion kernels and the experimental results shows the proposed method has powerful generalization ability, we believe it can well generalize to arbitrary kernels.
  4. The estimated kernels are shown in the supplementation of the main paper, you can find it in the CVF.
  5. Yes, we assume the spectral response function is konwn in this work because it can acquire easily, it can be seem as kind of regularization in this work. If you curious about the HSI SR fusion problem when spectral degeneration function is unknown, a work of ours to address this problem will come soon.
  6. The MHF-net is the only one supervised method which need to be trained, utilizing noisy data to train the MHF-net might improve is performance to deal with noisey data but can not improve its generalization ability on different spatial degenerations.
JingyunLiang commented 4 years ago

Thanks for your quick reply. Nice job! The generalizability to different kernels is amazing.

What do you think of the importance of the regularization of spectral response function, large channel number (31-dimensional data share the same kernel), and the design of the adaptation model? Did you try relevant ablation study?

Btw, where can I download the four kernels in Fig. 5 of the paper?