search

alexlee-gk / video_prediction

Stochastic Adversarial Video Prediction
https://alexlee-gk.github.io/video_prediction/
MIT License
303 stars 65 forks source link

what is the difference between Bair action free and action conditioned #33

Open NagabhushanSN95 opened 5 years ago

NagabhushanSN95 commented 5 years ago

You mention about 2 kinds in BAIR dataset: action-free and action-conditioned. What is the difference between the two? In the download script, there is option to down bair only. It doesn't distinguish between action-free and action-conditioned. But while downloading pretrained models, there are 2 different models for action-free and action-conditioned. So, I'm assuming the difference is related to how the models are trained. Is that correct? If so (or if not also), what is the difference between the two?

I generated some sample videos using both on savp model. I couldn't find any difference.

sthoduka commented 5 years ago

The action-conditioned model is described in Appendix B.3 of the paper:

...the video prediction model is now also conditioned on a sequence of actions a_0:T−1, in addition to the initial frames. The generator network is modified to take an action a_t at each timestep, by concatenating the action to the inputs of all the convolutional layers of the main network

i.e. the robot's state is also used to condition the video prediction, not just raw video.