More joints?

[robinpaulson]

Great to see this project, it appears to be the only complete free software stack for doing skeleton tracking. It's stable enough that I've been using it for a while now in a performance art project, very few complaints. What I would like to see though is more joints incorporated - is there any likelihood of this happening?

[joaquimrocha]

Hi Robin,

The plan is to add more joints indeed, there is an attempt of adding legs but it is still very experimental so it hasn't been included yet.

BTW, what's that art project about? Do you have any link to it I could check out? I love that kind of stuff!

[criverc]

Dear Joaquim, I work for a company in Spain named Envitel, we are participating in a research European project where we are building a walker for old people. This walker is supposed to detect the posture of the old person using it so it can determine whether he is performing a series of exercises correctly, also to detect possible health problems related to bad posture. I've been looking at your skeletization software which looks very promising but it is not quite there yet for us. We are missing the lower body detection. Have you made any progress on that? Could I collaborate in any way? Also If you are interested I ported your test-kinect example to kinect v2, basically I mirrored your repo, modified it and made it available here:

git clone http://farside.envitel.com:8080/git/Skeltrack.git

Regards!!

[criverc]

Hi @joaquimrocha ! I have merged the cloud point and skeleton views in the kinect.c demo. Now it is possible to switch between one, the other or both by hitting SPACE. I modified a bit my own repo so all my changes are now in branches. This particular changed is in a branch named criverc_WIP. I find that mixing views is useful for debugging. By doing this also to kinect2 demo I noticed that there is something wrong with my port because the head is way below where it should be. Everything works well with kinect 1 demo, as it should. In fact I cleared my office space and it now works well, indeed I had too many objects lying around and confusing the algorithm.

[joaquimrocha]

Hi @criverc ,

It's true Skeltrack needs to be improved to match more use cases and one due feature is detecting the lower body. @iaguis has made some experiments with that but it's probably not stable yet: https://github.com/iaguis/Skeltrack/tree/legs

If you have the time to hack on it and improve it, your participation is very welcome! I keep saying I will get some time to merge a few improvements from other people but so far I haven't been able to. In December I might have a little more time.

Cheers,

[criverc]

Hi @joaquimrocha , I looked at that code. It is true, it is not stable at all. It crashes with a segmentation fault after the addition of feet tracking. Not before. I found out that the problem seems to be when running the dijkstra_to function with a source (source_feet) that has i or j larger than width or height respectively, that assigns the distance array to a position that is out of bounds. This happens after source_feet gets assigned to centroid. And centroid gets its i and j set from the function convert_mm_to_screen_coords. I noticed that this function sometimes gives that result, i.e. i and j to what seems to be out of bounds. But I don't really understand where that formula comes from:

i = round (width / 2.0 + x / ((gfloat) (z + MIN_DISTANCE) * SCALE_FACTOR dimension_reduction * width_height_relation)); j = round (height / 2.0 + y / ((gfloat) (z + MIN_DISTANCE) * SCALE_FACTOR dimension_reduction));

Some help would be greatly appreciated!

Regards,

[joaquimrocha]

Hi @criverc ,

The calculations were originally taken from the libfreenect wiki, I believe this must be the article: http://openkinect.org/wiki/Talk:Imaging_Information

It's been a while since I tried that branch and it was @iaguis who developed it so it'd take me a little while to understand what's going on and what's wrong.

As I said, I plan to merge a few improvements that people have contributed.

As for the legs, just as we find the upper body, finding the legs should be possible but it takes some time to try different approaches and see what's the best. One problem is how to neglect the floor. For that we can maybe subtract the background every few seconds (I don't like this so much) or we can limit the length of the limbs based on the estimated size of the body (might give different results for different body types but I actually prefer this one as it will work also when the user is grabbing something e.g. a stick).

It'd be great to see a video of Skeltrack working with the Kinect 2, if you have one, please share it :)

[criverc]

hello @joaquimrocha, I have some trouble with the kinect2 port. The skeleton does not show up right, I have printed the average distances and compared to the ones obtained with kinect1 version to make sure I am getting the correct scale for the values of the depth map. They are pretty much the same. However I am looking at the formula to get x,y from i,j,z according to http://openkinect.org/wiki/Imaging_Information and it seems to me that the constants Dmin and scale_factor must be dependent on the device, i.e. those values must be good for kinect1 but not for kinect2. Using those equations I solved for Dmin and scale_factor and have a way to calculate it from image measurements, say look at a 1m stick at two different distances and calculate from the i,j, and z values measured, as well as width and height. However it is kind of hard to do the appropriate experiment to obtain the values. Do you know how they were obtained for kinect1? Is there an easier way to do it? I do not have a video yet! I wanted it to work right before I do it.