search

pedronunoleite / NEON

A lightweight Convolutional Neural Network (CNN) for depth estimation, designed for balancing both accuracy and inference times.
1 stars 1 forks source link

Input images #1

Open LulaSan opened 3 years ago

LulaSan commented 3 years ago

Hi! Thank you for sharing your impressive work! It is not clear to me which kind of input you are taking from KITTI Dataset, I have understood, reading the eigen_spli_train.txt file, that the last is the ground truth, but the first and the second seems to be the same! I thought they were the left and right frames of a stereo camera, am I wrong? Thank you!

LulaSan commented 3 years ago

Ok reading carefully I have understood that the second term is the magnitude of motion. So, is it suitable for real-time applications? Shall it need an optical flow algorithm to run in parallel?

pedronunoleite commented 3 years ago

Hello @LulaSan! Sorry for my delayed response.

The network does receive an RGB image and the magnitude of motion as inputs. You would indeed need an optical flow algorithm to calcutate the flow vectors between a stereo pair of images, and then take its magnitude and feed it to the network. We have obtained these results with the optical flows generated via the FlowNet 2.0 network. We also tried a simple opencv-based Brox optical flow algorithm, which lead to slightly worse results (since the quality of the input is also significantly lower).

The whole idea was to build a low-latency network that could be useful in real-time applications, so I wanna say yes to your question, but please bear in mind that the inference times are relative to the hardware in which the network is running.