search

shicaiwei123 / TGRS-MSHNet

Code for MSH-Net: Modality-Shared Hallucination with Joint Adaptation Distillation for Remote Sensing Image Classification Using Missing Modalities
MIT License
5 stars 0 forks source link

Why need to train single modality baseline? #2

Open ALEX13679173326 opened 1 year ago

ALEX13679173326 commented 1 year ago

Nice Work!! But I have a doubt that why not to directly modal fusion teacher? There is no weight loading process in huston2013_multi_train.py, so why train single modality baseline ??

shicaiwei123 commented 1 year ago

Hello, thanks for your attention.

Here, the single modality baseline is presented to support the motivation of the work. In other words, when the modality is missing (such as MS in HS+MS), why not directly use the single-mode model (HS model) for inference, but need to design an auxiliary structure, such as DHADNet, or MSHNet, to introduce additional modality information?

Here, the performance of MS in MSHNet or DHADNet is better than that in the unimodal, demonstrating the necessity of using auxiliary modalities.