No data on raspberry pi

[xxorde]

I noticed other people had similar problems, but I did not find an issue. If there is already one, please delete this.

I can compile freenect without any problems. When I compile a small c program (works fine on other machines) I don't get any data from “freenect_sync_get_depth”. Non of the examples works.

The program simply stops and never finishes this function.

In other words, in the following example “done” is never printed.

short *depth = 0;
uint32_t ts;
FILE *fp;
int ret;

printf("get depth....");
ret = freenect_sync_get_depth((void**)&depth, &ts, 0, FREENECT_DEPTH_MM);
printf(" done\n");

How can I find an fix the problem? Where to start?

[xxorde]

In the file "libfreenect_sync.c" on line 315 the command "pthread_cond_wait(&buf->cb_cond, &buf->lock);" never stops waiting.

static int sync_get(void **data, uint32_t *timestamp, buffer_ring_t *buf)
{
        pthread_mutex_lock(&buf->lock);
        // If there isn't a frame ready for us
        while (!buf->valid)
                pthread_cond_wait(&buf->cb_cond, &buf->lock);
…

So it looks like, there is no frame ready for us...

[zarvox]

The sad fact here is that the combination of libusb, the Pi's underwhelming CPU, and the Pi's disappointing USB controller can't handle the data rate of the isochronous streams at the needed latency and without dropping too many packets, so no full frame is ever acquired.

You could try modifying the library to tolerate more dropped packets (see line 133 of src/cameras.c), but you're more likely to get corrupted data or frames with large swaths of missing data.

[xxorde]

That does not sound good. But on my pi I don't get so far. On line 102 (src/cameras.c) hdr->magic[0] and hdr->magic[1] both have the value zero.

if (hdr->magic[0] != 'R' || hdr->magic[1] != 'B') {
        FN_LOG(l_notice, "[Stream %02x] Invalid magic %02x%02x\n",
               strm->flag, hdr->magic[0], hdr->magic[1]);
        return 0;
    }

[Stream 80] Invalid magic 0000

[psiniemi]

I'm trying the same with no luck. My test code, tcpdump of usb traffic, vmstat -s output and logging output at https://github.com/psiniemi/kinectOnRaspberryPi

Would there be a way to ask less data or something like that?

I can also provide any other useful data if someone wants to help...

[xxorde]

In my opinion, the best way would be to remove libusb abstraction layer by writing a kernel driver. This should do the job and bring better performance. Of course this is a linux only way.
And I hope we do not need to do this.

[xxorde]

Ok, I started to build a simple native driver which I hope runs on the pi. I am not really experienced and don't made to much progress. I too a skeleton driver which hooks up to the kinect. I can send messages with usb_control_msg() and get the answer, but stuck on sending custom packets like shown in the Protocol_Documentation. For example the one that should be replied by "05 00 01 00".

If someone wants to help me or just giving me a little hint, please feel free to do so!

[ray4510]

Great xxorde. I have been building a robot doing the software dev on a pc, now I am very disappointed to find that the RPI does not work with the kinect. please fix it!!

[piedar]

You might try @wizgrav's Chunk processing feature #351.

Testing indicates that the overhead of getting a 320x240 frame, unpacked and converted to mm(passing a lookup table through dev->user_data) is roughly the same as getting the packed 640x480 frame.

See chunkview.c for an example.

[ray4510]

Thanks for responding, but I couldnt get my noobie programmer head around it. I have been using python, and cant grasp C just yet,( i got compile errors related to gcc config paths and i have no room in my brain for that yet). I understand what it does, but i dont understand how to make that a factor with the python wrappers.

On Sun, Jan 19, 2014 at 12:17 PM, Benn Snyder notifications@github.comwrote:

You might try @wizgrav https://github.com/wizgrav's Chunk processing feature #351 https://github.com/OpenKinect/libfreenect/pull/351.

Testing indicates that the overhead of getting a 320x240 frame, unpacked and converted to mm(passing a lookup table through dev->user_data) is roughly the same as getting the packed 640x480 frame.

See chunkview.chttps://github.com/OpenKinect/libfreenect/blob/master/examples/chunkview.cfor an example.

— Reply to this email directly or view it on GitHubhttps://github.com/OpenKinect/libfreenect/issues/338#issuecomment-32713483 .

הכנס טקסט לתרגם כאן

[piedar]

Oh, I didn't know you're using Python. That feature isn't available in the Python wrapper yet.

[wizgrav]

Chunked processing will certainly alleviate a lot of bandwidth pressure but I'm afraid that this is not the culprit here. The problem is in the rpi itself. The usb controller that comes in the soc is of very low quality and combined with the low quality of the kinect itself, frame drops and failure ensue. I had a pi and spent lots of time on their forums before I realized that it's a dead end. If you're looking for a cheap arm board to run on, I suggest that you look somewhere else, like the cubieboards or this one https://www.olimex.com/Products/OLinuXino/A10/A10-OLinuXino-LIME/open-source-hardware

[xxorde]

Another problem is the libusb abstraction layer. I hope a native linux driver, which runs in kernel not in user space would do the trick! That's why I'm working on it.

By the way, I can send packages to the cam that look exactly like the once from the M$ driver in wireshark. The responses start with the right magic, but do not contain the right data. Maybe my init stuff doesn't work right.

send: 47 4D 02 00 03 00 00 00 00 00 01 00 receive: 52 42 01 00 03 00 00 00 00 00

I should have revived something containing 05 00 01 00. The normal depth init packets doesn't get the right response either, but I am on it.

[ray4510]

If i follow the conversation correctly, it's time to trade in my pi for a SunXI or something like that. I forgot there were options now. can someone help me recalibrate my google-eye, and tell me how I narrow the search on a quality board. I would want it to run Debian, and have ezmode gpio. I'd like to transfer what I know now, over to the new hardware and expect the same rules would apply (ie debian, vncsever, lirc etc NOT ARCH!) I dont know enough to know what to expect with different hardware like this, and I'd hate to find out six months down the road I'd become a Mac user.

[piedar]

@ray4510 If I were picking out a pi-like board, I would go with the BeagleBone Black. Unfortunately, I have no idea how well it works with Kinect, so further research is required.

[wizgrav]

I haven't tried the beaglebone but I had good results coming out of a beagleboard in the past so I think it should be adequate. My most recent purchase is the odroid u3 which is a tiny bit more expensive($60) but it's a quad core A9 @1.7 Ghz with 2GB RAM so it has very good value/price. I've also tested freenect succesfully on a pandaboard, cubieboard, olimex a10, samsung chromebook (Cortex A15). Basically the only board it failed was the Pi, and it's the most successful among all the arm boards. It's utterly crappy and a testament that you can always count on hype over quality

On Mon, Jan 27, 2014 at 8:24 PM, Benn Snyder notifications@github.comwrote:

@ray4510 https://github.com/ray4510 If I were picking out a pi-like board, I would go with the BeagleBone Blackhttp://beagleboard.org/products/beaglebone%20black. Unfortunately, I have no idea how well it works with Kinect, so further research is required.

— Reply to this email directly or view it on GitHubhttps://github.com/OpenKinect/libfreenect/issues/338#issuecomment-33404289 .

Yannis Gravezas Founder@Intrael http://www.intrael.com Athens, Greece +30 6947931815

[ray4510]

I've been looking at the olimex(allwinner) a-20, but i can't find anything on the net with people using openkinect on it. I want to assume it's because it just works! but that's how I ended up with an RPI in stead of a quality ARMy. can anyone verify the kinect will work with the olimex a20?

[wizgrav]

I've used the kinect succesfully with a cubieboard a10. The a20 is supposed to have the same USB components in it so I assume it should also work. Unfortunately I don't own that particular boards but I don't think you'll have a problem. Mind that the pi is notorious for it's crappy usb, every other arm board I've tried worked ok

On Mon, Feb 17, 2014 at 5:21 PM, ray4510 notifications@github.com wrote:

I've been looking at the olimex(allwinner) a-20, but i can't find anything on the net with people using openkinect on it. I want to assume it's because it just works! but that's how I ended up with an RPI in stead of a quality ARMy. can anyone verify the kinect will work with the olimex a20?

Reply to this email directly or view it on GitHubhttps://github.com/OpenKinect/libfreenect/issues/338#issuecomment-35290664 .

Yannis Gravezas Founder@Intrael http://www.intrael.com Athens, Greece +30 6947931815

[xxorde]

Success!

I have a running kernel driver! After some cleanup I can make my repo public. Because I also had to make small changes to gspca some more work is needed to make it ready for the kernel.

[arunchettoor]

Hi all ! I am working on Beaglebone black running 3.8.13-bone37 kernel. I am also facing the same problem where the code will be stuck at the position shown below:

while (!buf->valid)
                pthread_cond_wait(&buf->cb_cond, &buf->lock);

I tried in RPi and not succeeded and turned to Beaglebone Black to try my luck, because many people have successfully tried libfreenect with beagle board.

Great to hear @xxorde got it with RPi.. Congrats!!! Can you please explain the process so that i can also try the same in Beaglebone Black and let you know the results.

[xxorde]

Here is the native driver https://github.com/xxorde/librekinect it works well on my raspberry pi. If I write the data to my sd card I have 5% cpu usage, if I keep them in ram 2%.

I hope you like it.

@arunchettoor Have you got libfreenect running?

[floe]

@xxorde very impressive, you should definitely talk to the gspca devs to get this included into the mainline kernel at some point.

[xxorde]

Am 15.04.2014 17:26, schrieb Florian Echtler:

@xxorde very impressive, you should definitely talk to the gspca devs to get this included into the mainline kernel at some point. Thx, I already did. Next step is to change gspca in a way that I can get the second isoc endpoint, but still be compatible to the old api.

My version of gspca.c works for the kinect but for nothing else else ;)

---

Reply to this email directly or view it on GitHub: https://github.com/OpenKinect/libfreenect/issues/338#issuecomment-40495395

[piedar]

@xxorde Nice work! What's the best way to get frames from userspace? v4l2?

[xxorde]

I am not sure what you mean. I use v4l so the kinect act like any other "webcam". You get a new /dev/videoX

For example: camorama -d /dev/video0 vlc v4l:///dev/video0

I like to use OpenCV because I do not want do visualize the information, I want to use it for computation.

[piedar]

I was wondering how to open /dev/videoX from code. I've found an example of v4l-utils that appears to answer my question.

[arunchettoor]

@xxorde : Thank you so much for posting this. Loaded back everything to try in RPi. Everything was clear in manual unless the unzip of the git clone gz file. But I have problem at insmod gspca.ko showing below the output of "make load"

sudo sh unload_drivers.sh
Error: Module kinect is not currently loaded
Error: Module gspca is not currently loaded
Error: Module gspca_kinect is not currently loaded
Error: Module gspca is not currently loaded
Error: Module kinect is not currently loaded
make[1]: Leaving directory `/home/pi/Desktop/arun/librekinect/librekinect'
sudo modprobe videodev
sudo insmod gspca.ko
Error: could not insert module gspca.ko: Invalid module format
make: *** [load] Error 1

"dmesg | tail -10" gives:

[ 4041.074163] gspca_main: v2.14.0 registered
[ 4041.077283] gspca_main: kinect-2.14.0 probing 045e:02ae
[ 4041.079860] usbcore: registered new interface driver kinect
[ 4504.211681] usbcore: deregistering interface driver kinect
[ 4504.419876] gspca: no symbol version for module_layout
[ 4547.199514] gspca: no symbol version for module_layout
[ 5414.106806] gspca: no symbol version for module_layout
[ 5748.719800] gspca: no symbol version for module_layout
[ 6565.750570] gspca: no symbol version for module_layout
root@raspberrypi:/home/pi/Desktop/arun/librekinect/librekinect# uname -r
3.10.25+

Please help with this to try the code..

[xxorde]

Am 17.04.2014 02:15, schrieb Arun Krishnan Chettoor:

sudo sh unload_drivers.sh Error: ... Ignore these, unload_drivers.sh is super dirty and just tries to unload everything which could get in the way even if its not there.

make[1]: Leaving directory `/home/pi/Desktop/arun/librekinect/librekinect' sudo modprobe videodev sudo insmod gspca.ko Error: could not insert module gspca.ko: Invalid module format make: *\ [load] Error 1

"dmesg | tail -10" gives:

[ 6565.750570] gspca: no symbol version for module_layout This indicates you have compiled the module against a different version / configuration of the kernel.

@xxorde : Thank you so much for posting this. Loaded back everything to try in RPi. Everything was clear in manual unless the unzip of the git clone gz file.

Oh, I forgot to post the command to unpack the kernel, I am going to add it.

tar xfvz rpi-3.10.y.tar.gz

Are you sure, you used your current config, by this:

gzip -dc /proc/config.gz > /usr/src/linux/.config

Just try to use the kernel from the source you downloaded, I just changed the readme.

sudo -s
cd /usr/src/linux
make
make modules_install
cp /usr/src/linux/arch/arm/boot/zImage /boot/linux-3.10.y
echo "kernel=linux-3.10.y" >> /boot/config.txt
reboot

I hope you can load your modules after this!

[arunchettoor]

@xxorde I done the kernel compiling as you said and successfully loaded the modules. Thank you. But I need to get the depth values as the function: freenect_sync_get_depth((void**)&data, &timestamp, index, FREENECT_DEPTH_MM) So how can i communicate with the module to get the depth output . It will be so useful if you post an example code in C like the wrappers of libfreenect to just get values from the modules.

[xxorde]

@xxorde I done the kernel compiling as you said and successfully loaded the modules. Thank you. But I need to get the depth values as the function: freenect_sync_get_depth((void**)&data, &timestamp, index, FREENECT_DEPTH_MM) No, you don't! At this point you do not need freenect_sync_get_depth or any other component from libfreenect any more. You can simply get your depth data from "/dev/video0". What do you want to do with the depth data? Simply look how you can get data from a video device, linux has a standard (v4l) which I use. Nearly every software supports it. So how can i communicate with the module to get the depth output . It will be so useful if you post an example code in C like the wrappers of libfreenect to just get values from the modules. There are a lot of examples, try the search words "linux webcam", "v4l", "/dev/video0".

Here is an example with OpenCV in C++ http://henryhermawan.blogspot.de/2007/09/capturing-frames-from-usb-camera-using.html

Here is one with python on the raspberry: http://tothinkornottothink.com/post/59305587476/raspberry-pi-simplecv-opencv-raspicam-csi-camera

If you don't like external frameworks and libraries you can use fopen() and process the data yourself.

If you have your frames, every pixel hast a 10 bit value which indicates the distance from the camera. Higher value means greater distance, zero means not able to compute (mostly shadow or to close).

I am glad that you got it! :)

[xxorde]

It's better when we discuss further topics about the native driver where they belong: https://github.com/xxorde/librekinect I close the issue for now.

[MarkTillotson]

With vanilla libfreenect I found that the lack of USB oomph means you can only use the Raspberry Pi with the Kinect by limiting to sensing one camera at a time ie only enable one of depth or rgb streams, switching between them if necessary. If you only want depth information this is bearable. If real-time RGB point clouds are wanted, no dice.

It may be that using the smallest resolution option allows both streams to co-exist, but I haven't tested this.

[AIWintermuteAI]

It's an old issue, but just in case anybody encounters it, in search of solutions - I recently wrote an article about using Kinect 360 and RTAB-MAP ROS with Rasberry Pi 4, with some tweaks it does work now! https://www.hackster.io/dmitrywat/rgb-d-slam-with-kinect-on-raspberry-pi-4-ros-melodic-ace795