search

vincent-leguen / PhyDNet

Code for our CVPR 2020 paper "Disentangling Physical Dynamics from Unknown Factors for UnsupervisedVideo Prediction"
MIT License
179 stars 50 forks source link

Query regarding concatenation #33

Closed fnuabhimanyu closed 1 year ago

fnuabhimanyu commented 1 year ago

https://github.com/vincent-leguen/PhyDNet/blob/23a992d771c9eb1d32f52b1873a3c5625f1a8413/models/models.py#L279

In the model, it looks like the latent vector from the physics model and convLSTM are added. Should it have been concatenated along the channel? Also looking at Sec 2.2 of the Supplementary Material: Model architectures, it looks like its concatenated. Can you please clarify?

vincent-leguen commented 1 year ago

Hi, I confirm that the latent vectors from the PhyCell and the ConvLSTM are indeed added and not concatenated, as can be shown in Figure 2 a) and in the text at the bottom of page 3. Then the channel dimension remains the same between the last layer of the encoder and the first layer of the decoder (for example 64 for Moving MNist).

fnuabhimanyu commented 1 year ago

Thanks Vincent. This helps.