search

coxlab / prednet

Code and models accompanying "Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning"
https://arxiv.org/abs/1605.08104
MIT License
759 stars 259 forks source link

Multi-step prediction ValueError: loading weight file containing 1 layers into a model with 3 layers. #58

Closed severin1992 closed 5 years ago

severin1992 commented 5 years ago

Hi everyone, first of all huge thanks to @bill-lotter for providing this code! I'm training the PredNet on weather data and t+1 predictions work pretty well. Unfortunately, I'm struggling with training/ evaluating the multi-step prediction network. Running python kitti_extrap_finetune.py seems to work. When I try to evaluate my network, I always receive the following error message:

ValueError: You are trying to load a weight file containing 1 layers into a model with 3 layers.

My evaluation code looks very much like the one from @abe- https://github.com/abe-/landscape-prediction/blob/master/prednet/evaluate.py where batch_size = 10, nt = 15 and extrap = 10.

Interestingly I can manage to evaluate the pretrained model. What's interesting though is that when loading prednet_kitti_weights-extrapfinetuned.hdf5 the data_format is set to channels_first. In the t+1 setting it is channels_last. Here one example from a map over Europe. If everything works, from frame 10 on it should only be extrapolations. plot_79

severin1992 commented 5 years ago

I assume this might have something to do with #9 and #11 and maybe https://github.com/keras-team/keras/issues/10417. I'm running on Keras/2.2.4-GPU-Python-3.6.6 I also saw that when running python kitti_extrap_finetune.py there was no new model saved (prednet_kitti_model-extrapfinetuned.json). Maybe because I ran out of processing time. I'll try again and see whether the model is saved then. Is there someone else who faced this issue? Or can maybe someone upload his/her model .json file? Maybe @kikyou123 or @robbiewu008. Thanks in advance

severin1992 commented 5 years ago

It works now. For everyone who might face the same problem later: The new saved prednet_kitti_model-extrapfinetuned.json differs from the original one. So, just make sure to run kitti_extrap_finetune.py until the end, then a new .json file will be created.

Roy-Rupak commented 4 years ago

Hi @severin1992 While trying to run kitti_extrapfinetune.py, I got the following error "ValueError: Error when checking input: expected input_1 to have shape (10, 3, 128, 160) but got array with shape (10, 128, 160, 3)". Did you face any such issue? Also, can you share the code that you used for training the multi-step prediction? Thanks.

jawa0 commented 3 years ago

@Roy-Rupak There's an option somewhere in the PredNet code to specify whether image channels come first or last. So for instance an RGB (3 channels) 160 col by 128 row image might be either (3, 128, 160) or (128, 160, 3). You need to set the option appropriately.