FlyThings3D as inputs leads to out of memory

[chuchienshu]

run-flownet-many.py models/FlowNet2/FlowNet2_weights.caffemodel.h5 models/FlowNet2/FlowNet2_deploy.prototxt.template flythings3d.txt where flythings3d.txt file just including six pairs of .png picture that from FlyThings3D RGB_cleanpass. I run it on ubuntu16.04 with two Titan xp.But below is what I got!

Processing tuple: ['/home/chuchienshu/caffecode/RGB_cleanpass/left/0006.png', '/home/chuchienshu/caffecode/RGB_cleanpass/right/0006.png', '/home/chuchienshu/caffecode/RGB_cleanpass/fly6.flo']
Network forward pass using models/FlowNet2/FlowNet2_weights.caffemodel.h5.
WARNING: Logging before InitGoogleLogging() is written to STDERR
F1024 17:04:35.780727  3358 syncedmem.cpp:56] Check failed: error == cudaSuccess (2 vs. 0)  out of memory
*** Check failure stack trace: ***
Aborted (core dumped)

When I run the same command replaced flythings3d.txt with sintel_clean.txt, which including some Sintel pairs.Everything is fine! I have no idea about it. Any suggestion will help. Thanks a lot!!

[nikolausmayer]

Are there any other GPUs in your machine, possibly one with little VRAM that might be used by mistake? Or are there other programs or users that are using the GPUs at the same time? I can run the FlyingThings3D example data on my GTX 970 (4GB) without a problem.

[chuchienshu]

Thanks for you reply so much! According to what you said, I convinced this is my mistake. Another question. That's Data Augmentation, which you mentioned at Flownet1's paper but Flownet 2 do not, although you used it ,too. I post part of the statement at Flownet1's paper below.

Specifically we sample translation from a the range
[−20%, 20%] of the image width for x and y; rotation from
[−17 ◦ , 17 ◦ ]; scaling from [0.9, 2.0]. The Gaussian noise
has a sigma uniformly sampled from [0, 0.04]; contrast is
sampled within [−0.8, 0.4]; multiplicative color changes to
the RGB channels per image from [0.5, 2]; gamma values
from [0.7, 1.5] and additive brightness changes using Gaus-
sian with a sigma of 0.2.

Actually, I don't know weather all Augmentations are processed together or processed respectively. Given the change is so great，If all Augmentations get together, the input images are going to be very different from reality pictures. so ,how come it get better result?

[nikolausmayer]

As far as I know, we used the same augmentations for FlowNet2 as for FlowNet1. The augmentations are all processed together. Yes, this can lead to big changes in the images. Considering that optical flow is a low-level task, we think that "realism" is less important than training the network to be robust; and for that, we want as diverse training data as possible. The un-augmented data seems to not contain quite enough variation for this.

[nikolausmayer]

Let's take frame 0008 as an example:

• The "into_future" flow 0008 is valid for 0008 -> 0009
• The "info_past" flow 0008 is valid for 0008 -> 0007

The "left" and "right" split exists because the dataset contains stereo images for every frame. If you just want optical flow, you can just e.g. ignore every "right" subfolder.

[chuchienshu]

I just find it's stereo images, Thanks for your patience and kindness.