search

TmacMai / ARGF_multimodal_fusion

codes for: Modality to Modality Translation: An Adversarial Representation Learning and Graph Fusion Network for Multimodal Fusion
45 stars 11 forks source link

loss question #5

Open ffredd opened 1 year ago

ffredd commented 1 year ago

I'm here again, ha! This code can indeed converge and get good accuracy results. However, the validation set of this code only discusses the loss value of the classifier, not the loss value of the generator. And I output the loss value of the GAN generator and discriminator after the model converges, I find that they have no convergence. Does this affect the interpretability of the entire experimental process, or do you just focus on the final classification effect? I'm looking forward to ur reply!

TmacMai commented 1 year ago

Actually, GAN is difficult to be trained due to the mode collapse problem. It is possible that at some parameter settings, the loss of GAN does not coverage. It might help to change the plain GAN into W-GAN, or enlarge the weight of GAN's loss.

ffredd commented 1 year ago

Yes, I used gan on my dataset, and found that even though loss did not converge, the final fusion classification effect was still very high. I think the gan encoder at this time just played the effect of an encoder. I see that your code is just basic gan, can it also be converged?