theislab / multimil

Multimodal weakly supervised learning to identify disease-specific changes in single-cell atlases
https://multimil.rtfd.io/
BSD 3-Clause "New" or "Revised" License
19 stars 3 forks source link

Add cycle consistency loss #5

Closed awakeenderman closed 4 years ago

awakeenderman commented 4 years ago

The problem is way harder in unpaired setting. The model should be regularized to be able to find the correct mapping of modalities. Without regularization, the integration of modalities in the latent is not meaningful (as tested). One way to overcome this issue is to add cycle consistency to the model as follows:

  1. Take x_i (the feature matrix in domain i) and encode it to become z_i.
  2. decode z_i with condition j to become r_ij (the reconstructed x_i through the decoder of modality j). Here we have converted the feature matrix x_i from domain i to domain j.
  3. Encode the r_ij to reach z_ij.
  4. The distance between z_i and z_ij should be minimum.

Tasks:

awakeenderman commented 4 years ago

In paired setting, it seems that the integration loss will do the job and having cycle consistency makes no difference.