Ideas for feature networks

[MrWhatZitToYaa]

• [ ] Pack frames of video in an array -> 3D tensor -> Run kernel through the whole depth of the Video
• [x] RNN + CNN
• [ ] Try to reduce dimensionality with VAE

[psaegert]

• [x] Encode videos separately with the same CNN (+ FCNN / LSTM / Attention) network and concatenate their feature vectors
• [ ] After encoding with a feature detector, append a few fully connected layers and add the SINDy constraint that encourages the features to reflect the physics behavior (best case: learn to extract the trajectories from the video)
• [ ] Use algorithmic stereo reconstruction to extract the 3D trajectory for use as the condition

[MrWhatZitToYaa]

• [x] pretrain on trajectories and then further train on videos after modifying the network --> mild-breeze-113

[MrWhatZitToYaa]

• [ ] add 2 CNNs (for each camera) and see if the performance gets better in comparison two 1 -> bumbling-pine-122 (double) ->azure-energy-123 (single)