search

goodrobots / vision_landing

Precision landing using visual targets
GNU General Public License v3.0
202 stars 71 forks source link

Not receiving any output video frames #79

Closed petergerten closed 7 years ago

petergerten commented 7 years ago

I am using a test setup on Ubuntu 16.04. When I start vision_landing it successfully connects to my ArudCopter SITL. The video device is accessed (I tried several different cameras - the LED comes on and no log error). But there is no output. I tried 'window', I tried file output (I get some 5kb in there and that's it) and I also tried a gstreamer pipe. No luck. I enabled verbose mode but for some reason all log output looks just fine:

2017-07-22 09:45:10,191 INFO Launching track_targets with arguments:/home/peter/vision_landing/track_targets -d TAG16h5 -o /var/tmp/vision.avi -w 320 -g 240 -f 30 -v -z 11:0.446,16:0.204,19:0.104,23:0.047 /dev/video0 /home/peter/vision_landing/calibration/mycam.yml 0.5 2017-07-22 09:45:10,193 INFO Connecting to drone on: udp:localhost:14550 2017-07-22 09:45:11,498 INFO Connected to drone 2017-07-22 09:45:11,498 INFO Setting flight controller parameters 2017-07-22 09:45:11,499 INFO track_targets: info:FoVx~51.8032:FoVy~40.1442:vWidth~320:vHeight~240 2017-07-22 09:45:11,499 INFO track_targets: info:Size Mappings:11=0.446, 16=0.204, 19=0.104, 23=0.047, 2017-07-22 09:45:11,499 INFO track_targets: debug:Marker History:15 2017-07-22 09:45:11,499 INFO track_targets: debug:Marker Threshold:50 2017-07-22 09:45:11,499 INFO Entering main tracking loop

peter@ubuntu:~/vision_landing$ ./vision_landing APM:Copter V3.6-dev (401a6f53) Frame: QUAD

fnoop commented 7 years ago

Huh, that's weird. At the very least it should push the frames through to a window or a file. For 'window' are you running it on a local machine with a display? What hardware are you running on?

petergerten commented 7 years ago

I run it on a "local" x86 system. Inside a vmware VMWare VM from a Linux host though. But the cameras all work just fine if I use gstreamer directly.

petergerten commented 7 years ago

After testing some stuff it seems that vision_landing somehow expects gstreamer 0.10 where my main gstreamer installation is 1.0. How ist that selected from within the application?

fnoop commented 7 years ago

Ah if you've compiled it against gstreamer 0.10 it may not work - I've never tried it. It was fully developed against gstreamer 1.0

petergerten commented 7 years ago

ok, I removed anything related to gstreamer 0.10 and rebuild. Still the same issue. Where would I find the reference to gstreamer? Is it in the vison_landing/tracktarget source or in opencv itself?

fnoop commented 7 years ago

So if you do: ldd vision_landing/track_targets |grep gst It should return something like:

[dev] [mav@maverick-joule ~]$ ldd /srv/maverick/software/vision_landing/track_targets  |grep gst
    libgstreamer-1.0.so.0 => /srv/maverick/software/gstreamer/lib/libgstreamer-1.0.so.0 (0x00007f91c345f000)
    libgstapp-1.0.so.0 => /srv/maverick/software/gstreamer/lib/libgstapp-1.0.so.0 (0x00007f91c3250000)
    libgstriff-1.0.so.0 => /srv/maverick/software/gstreamer/lib/libgstriff-1.0.so.0 (0x00007f91c3041000)
    libgstpbutils-1.0.so.0 => /srv/maverick/software/gstreamer/lib/libgstpbutils-1.0.so.0 (0x00007f91c2e0a000)
    libgstbase-1.0.so.0 => /srv/maverick/software/gstreamer/lib/libgstbase-1.0.so.0 (0x00007f91ba3e1000)
    libgstaudio-1.0.so.0 => /srv/maverick/software/gstreamer/lib/libgstaudio-1.0.so.0 (0x00007f91ba175000)
    libgsttag-1.0.so.0 => /srv/maverick/software/gstreamer/lib/libgsttag-1.0.so.0 (0x00007f91b9f3a000)
    libgstvideo-1.0.so.0 => /srv/maverick/software/gstreamer/lib/libgstvideo-1.0.so.0 (0x00007f91b9ca3000)

If you're not using maverick then it should just reference the same libraries but in /usr/lib

fnoop commented 7 years ago

If you look in src/CMakeLists.txt, you see it's just looking for OpenCV and aruco, not gstreamer directly. It uses gstreamer that is compiled into OpenCV. What OpenCV are you using? If it's ubuntu then it's almost certainly 2.4, which isn't tested/supported. I don't mind trying to get it to work however, but I do remember looking into it at the start and it's quite different code.

petergerten commented 7 years ago

Ah thanks! ok, there is something wrong here. Will try to fix. libgstreamer-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstreamer-0.10.so.0 (0x00007f1743974000) libgstvideo-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstvideo-0.10.so.0 (0x00007f1743757000) libgstapp-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstapp-0.10.so.0 (0x00007f1743549000) libgstriff-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstriff-0.10.so.0 (0x00007f174333c000) libgstpbutils-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstpbutils-0.10.so.0 (0x00007f1743118000) libgstbase-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstbase-0.10.so.0 (0x00007f173c0f9000) libgstaudio-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstaudio-0.10.so.0 (0x00007f173bc3d000) libgsttag-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgsttag-0.10.so.0 (0x00007f173ba05000) libgstinterfaces-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstinterfaces-0.10.so.0 (0x00007f1734591000)

petergerten commented 7 years ago

I actually uninstalled OpenCV 2.4 and build 3 from source as you suggested it in the project description. Seems I was not carefull about the installed gstreamer version though.

fnoop commented 7 years ago

Ah yes, it's a pain! I'm actually just finalising an Ubuntu VM release of maverick which has it all pre-compiled, installed and configured, if you just want to test it. Should be released tomorrow.

But, if you've uninstalled gstreamer 0.10 then you should be able to recompile opencv 3.2 and it should work.

petergerten commented 7 years ago

I rebuilt OpenCV 3.2 against gstreamer 1.0 and verified track_target is linked against the correct libs with ldd. But I still have exact the same beavhior as before.

petergerten commented 7 years ago

I also tried with test input and file output to make sure it is not a hardware / display issue: input=videotestsrc pattern=snow ! appsink and output=/var/tmp/vision.avi

Content of output file: https://pastebin.com/CvSucWpM

fnoop commented 7 years ago

I'll do a bit of testing and report back. I did an update a few days ago to correct the distance calculation and tested it with a video file input afterwards and it worked fine. Will test with a camera.

petergerten commented 7 years ago

Great, thanks! Any suggestion where I could insert debug output to narrow down the issue? I also reverted to a version without the updated distance calculation code but that did not change anything.

petergerten commented 7 years ago

I added some debug output and figured out that the main loop stops early due to stateflag==0. If I force stateflag=1 I get a video output. Will test further if the tracking works.

It seems that somehow my controlprocessing=False to force the processing loop on has no efect.

fnoop commented 7 years ago

Several commits recently should improve/fix this behaviour. Please reopen if still a problem.