search

coxlab / prednet

Code and models accompanying "Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning"
https://arxiv.org/abs/1605.08104
MIT License
759 stars 259 forks source link

How to generate multiple future frames from only a single input frame as a context ? #47

Closed rohanrc1997 closed 5 years ago

rohanrc1997 commented 5 years ago

Hello, I wish to know how can I make multi-step predictions from a single input frame, or more specifically, which parameters need to be tweaked to get the job done. I tried reducing 'td' down to 1, and recursively feeding the input back into the network (like the paper says). However, the output to the very first frame produces a dead output (gray image), which propogates and causes the loop predictions ahead to produce dead images again, without making any real predictions. Please correct me if I'm passing the input incorrectly. Hoping for a quick response. Many Thanks !

bill-lotter commented 5 years ago

Hello, An example of how the network can be fine-tuned for multi-step prediction is in kitti_extrap_finetune.py. You can try something like that with extrap_start_time = 1. It might not work that well with only one input frame, most likely it will just copy that frame for the most part.