Open lorenzoromagnoli opened 7 years ago
Hi Lorenzo,
Does this happen consistently on a specific frame? Have you tried looking at the frame values when it breaks?
It actually seems to happen on multiple frames. probably there are some issues with the image normalization procedure.
I temporarily solved it by getting rid of the +0.5, -0.5 transposition in the d_model.py
# sknorm_img = (img / 2) + 0.5
# resized_frame = resize(sknorm_img, [scale_net.height, scale_net.width, 3])
# scaled_gt_output_frames[i] = (resized_frame - 0.5) * 2
sknorm_img = (img / 2)
resized_frame = resize(sknorm_img, [scale_net.height, scale_net.width, 3])
scaled_gt_output_frames[i] = (resized_frame) * 2
This seems to have solved the issue and I managed to train and test the network.
However I'm not sure if my dataset has been prepared correctly. I trained the network with a video of me biking trough my neighbourhood; from the video-clips I take 3 images per second and use them for the training.
this is the result after just 1000 steps.
It seems to me that the generated frames are far closer to the 4th input frame that to the target expected image.
Do you think the frames of the input sequence are too far apart? or I should just let the machine train for way longer?
really appreciate your feedback on this.
I want to train the model with my data,but it didn`t success,may I ask ,how did you train this model with your data?
I'm trying to use your code but I get a strange error with the avg_runner. it seems like it has some issue with the size of some images. this is however very unlickly as I exported all the images from a video with ffmpeg.
any idea about why it breaks?