search

coxlab / prednet

Code and models accompanying "Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning"
https://arxiv.org/abs/1605.08104
MIT License
759 stars 259 forks source link

can not reduce model reduce #31

Open kikyou123 opened 6 years ago

kikyou123 commented 6 years ago

when i download your model, the Model MSE is 0.08421. But when I run the kitti_train.py and use the trained weights, I get the Model MSE 0.018259 this is my results in the test set: image Is the model in download_models.sh using the kitti_train.py?

bill-lotter commented 6 years ago

Yes the weights are trained using kitti_train. With the downloaded weights, are you missing a zero and it's 0.008421? Because that's about what I get when I use the tensorflow backend (with theano, I get ~0.007). Not sure why it seems to not train as well for some people. Easy first things to try are different optimizers/learning rates/momentum. Sometimes Rmsprop works better when adam struggles, so you can give that a try.

robbiewu008 commented 6 years ago

@kikyou123 I got the similar result with you, my env is tensorflow-gpu 1.2.1, keras 2.0.6 with cudnn v5.1 for cuda 8.0.

Model MSE: 0.017124 Previous Frame MSE: 0.021246

kikyou123 commented 6 years ago

@robbiewu008 It seems there is a bug in the code. in the kitti_train.py rrors_by_time = TimeDistributed(Dense(1, weights=[layer_loss_weights, np.zeros(1)], trainable=False), trainable=False)(errors) # calculate weighted error by layerr. It seems the layer_loss_weights can't load. you can change it image and try it again! I got the about the 0.007

robbiewu008 commented 6 years ago

@kikyou123 missing the Reshape, could you share it? image

kikyou123 commented 6 years ago

@robbiewu008 image

robbiewu008 commented 6 years ago

@kikyou123 still don't work.

image

kikyou123 commented 6 years ago

sorry, It is multi prediction code. You can change image ) 15 to 10

robbiewu008 commented 6 years ago

@kikyou123 It works now, thank you so much!

Model MSE: 0.006937 Previous Frame MSE: 0.021246

xukai8 commented 6 years ago

@kikyou123 Did you successfully train the model for muti-step prediction? I get the stable frames over time step.

bill-lotter commented 6 years ago

@kikyou123 It looks like there's an issue with TimeDistributed in Keras 2 where it overrides the initial weights of the layer to be wrapped (https://github.com/keras-team/keras/issues/8895). I changed the kitti_train.py code to account for this (https://github.com/coxlab/prednet/commit/9f6482e6e6038e02aa3dbd878bae23c7b3a5a28a). It should work now, but let me know if you find any other issues - thanks!

alvinxiii commented 6 years ago

@kikyou123 @bill-lotter hi. I tried run the multi prediction code. I barely see the differences from it. Is this the correct steps? This is my step:

  1. run kitti_train.py
  2. run kitti_extrap_finetune.py
  3. run kitti_evaluate.py