Blob detection not working

[BurningDog]

Firstly, thanks for a great app! It's been my first exposure to interfacing with the kinect on Mac, and KinectCoreVision is pretty easy to use - great work! :)

I understand that previously blob ("object"?) detection was working; however, try as I might, I cannot get it working. Fingers and fiducials work fine; I cannot get the Kinect to register any blobs.

I watched the calibration video you posted, and saw you how could do blob tracking, but I can't seem to get the current version to work.

[patriciogonzalezvivo]

Hello Burning Dog, thanks for the feedback. There are no more fiducials option. I switched it, for fingers. Because I´m not analyzing RGB images... and fiducials can be seen with depth images : ) . I just re-compile everything and "blobs" seams to work pretty well. I didn´t touch the "object" option from the original CCV. I understand that object it´s not blob detection. It´s a way that you could teach the CCV to detect specially kinds of contours... for example boxes or areas... something like that. But I don´t put to much effort on it. On the video I´m not showing any "object" activity. It´s all simple blobs... and then finger detection.

[BurningDog]

Ok, thanks. What I'm trying to do is blob detection (i.e. detect people in a scene) and then send the centre-of-mass x,y co-ordinates, along with z for depth, over TUIO. When I switch to "object" mode, no data gets sent, because no blobs are detected (in the "Debug information" window, it just reads '0,0,0' when I'm in object mode).

I'm reading through the code now to try to understand how it works. At the moment I'm thinking of using openCV to do blob detection, then doing a CoM calculation, and then grabbing the z co-ordinate at the CoM position.

Do you think this is a good approach? Could you point me at some example code?

[BurningDog]

Solution found! Turns out I was using an old binary that someone had emailed to me, in which the blob detection wasn't working. When I finally got KinectCoreVision compiled in xcode (which wasn't trivial, given a number of library linking issues that I finally got sorted) then it ran as I expected. w00t!

Blob detection still transmits as cursor events in TUIO, as opposed to blob events (I'd expected blob events), but for my project that's fine.

[BurningDog]

I'm not getting any depth data for the blob over TUIO - only x and y. investigates

[patriciogonzalezvivo]

Hello Burning Dog, it´s using TUIO 1.0 witch do not have Z. If you want to add it fell free to do it. You have to follow this tuio protocol specification: http://TUIO.org/?specification Defenetly you have to work round this area https://github.com/patriciogonzalezvivo/KinectCoreVision/tree/master/src/ofxKCore/src/Tracking To take de z from the depth when it´s making the contour analysis... store it in the blob... and then send it through TUIO following the link specification I mention before.

Then the community will be happy if you share you upgrades.

Thanks

[BurningDog]

Blob detection is done in Tracking.cpp and there doesn't look like any easy way to add depth data to that file. In addition, that file looks like it's from CCV and I'm not sure I want to make changes to it.

However, just before the x,y of the blob gets sent over TUIO, I can grab the z direct from the Kinect. Except I haven't a clue how to do that! Yet...!

[BurningDog]

Ok, I got this working :) Here's first commit message:

Added depth information to blob tracking, so we have the potential to send that over TUIO now. This is done by:

• Adding a new argument to ContourFinder::findContours which is ofxDepthGenerator& depth
• We then call getPixelDepth on the depth object, passing to it the x,y co-ordinates of the centre of the blob. The returned distance is in millimetres.

I'm not sure how to push my patch to you - here it is in the meantime: http://pastebin.com/RaP6qKUP

[BurningDog]

I've also noticed that in the previous version of KinectCoreVision I was using, the distance was measured between 0 and 255, but in this version it's done in millimetres - 4000mm max. I need a bit more distance! I'm not sure what the max distance is that the kinect can see up to - I'm using 10000mm (i.e. 10 metres). Much more useful!

Getting TUIO working should be easy enough. The current implementation uses 2Dcur and if we want to add depth to that we've got to use 25Dcur - which will break the normal way KinectCoreVision sends messages over TUIO. Maybe we should add an option checkbox to the GUI so that users can choose which TUIO message kind they want to send?

Anyway, I couldn't get it working. It seemed easy enough to edit TUIO.cpp and add in z in all the right places, but I couldn't get that data out in Flash (KinectCoreVision -> UDP -> a UDP-to-TCP convertor -> flash) and when I eavesdropped on the connection using tcpdump I got this:

sudo tcpdump -s 0 -A -X -c 1 -i lo0 port 3333 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on lo0, link-type NULL (BSD loopback), capture size 65535 bytes 23:03:25.715484 IP localhost.54139 > localhost.dec-notes: UDP, length 156 0x0000: 4500 00b8 0d2c 0000 4011 0000 7f00 0001 E....,..@....... 0x0010: 7f00 0001 d37b 0d05 00a4 feb7 2362 756e .....{......#bun 0x0020: 646c 6500 0000 0000 0000 0001 0000 0040 dle............@ 0x0030: 2f74 7569 6f2f 3235 4463 7572 0000 0000 /tuio/25Dcur.... 0x0040: 2c73 6966 6666 6666 6666 0000 7365 7400 ,sifffffff..set. 0x0050: 0000 00cc 3ede afd4 3f36 f23b 4484 4000 ....>...?6.;D.@. 0x0060: bcbe c000 3c84 4000 0000 0000 3a80 0fb6 ....<.@.....:... 0x0070: 0000 0020 2f74 7569 6f2f 3235 4463 7572 ..../tuio/25Dcur 0x0080: 0000 0000 2c73 6900 616c 6976 6500 0000 ....,si.alive... 0x0090: 0000 00cc 0000 0020 2f74 7569 6f2f 3235 ......../tuio/25 0x00a0: 4463 7572 0000 0000 2c73 6900 6673 6571 Dcur....,si.fseq 0x00b0: 0000 0000 0000 2b14 ......+. 1 packets captured 31 packets received by filter 0 packets dropped by kernel

I can see that the TUIO header is being set up correctly - 25Dcur - but I don't know how to read the message to make sure I'm sending it properly.

So I haven't patched KinectCoreVision yet. I don't actually need y info in flash, so I'm just sending through x and z. Fine for me, but doesn't solve this for anyone else.

[patriciogonzalezvivo]

Hello BurningDog, sorry for not replying very often last couple of mounth were superbussy to me. How are you? and how it´s your status? any problem or succes?