Add support for V4L2_PIX_FMT_GREY

[floe]

Some Video4Linux drivers (such as the experimental one for the SUR40 :-) directly provide an 8-bit greyscale image with format code V4L2_PIX_FMT_GREY. It's probably sufficient to just add this in https://github.com/mkalten/reacTIVision/blob/master/linux/V4Linux2Camera.cpp#L56 and https://github.com/mkalten/reacTIVision/blob/master/linux/V4Linux2Camera.cpp#L420 but I'm not familiar enough with portvideo to tell if that is indeed enough...

[norpol]

I would appreciate this also! If you need someone to test it on the SUR40, we have one lying around here.

[mkalten]

Sure, I'll take care of it, that part of the code needs some cleanup too apparently ... Norpol, actually we are looking for a SUR40 donation for these efforts!

[mkalten]

BTW, for backwards compatibility it also should be quite straightforward to add a MS byte tag tracker, using the existing libfidtrack segmentation data ;)

https://msdn.microsoft.com/en-us/library/ee804885%28v=surface.10%29.aspx

[norpol]

@mkalten I don't think we can donate it, since it was a donation by some stranger of our hackerspace in Vienna. But if you'd like to drop by, just tell me.

[mkalten]

sure, I'll check it out when I'm in Vienna next time ... but are you actually actively using it for anything else than a coffee table?

[norpol]

At the moment no one got enough time hacking around with it. :disappointed: I mean, no one is willing to use Microsoft development stuff - so we've installed Ubuntu which gave us a basic touch input. But we weren't able to figure out how to calibrate the whole thing under Linux and how to get the touch input into the browser.

[mkalten]

OK, I am back ;) the V4L2_PIX_FMT_GREY pixelformat is now handled by V4Linux2Camera let me know if this works for you ...

[norpol]

Super cool! I'm going to test it within a few days, lots of other things to do right now. Thank's a lot! ~~Update: Hi! I've finally managed to install Debian 8 on the table, at the moment I just get the error message no V4Linux2 devices found - need to investigate further... :) @floe I'm not sure if it's even supposed to work, at least when I'm on 3.16.0-4-amd64 I'm not having any /dev/video* device (though gnome3 'automatically' detects some touch input).~~

Oh nice it works! Just had to compile the recent kernel module. Hm, if I xinput disable $SUR40 the video stream just suddenly stops. Hm.

[floe]

+1 for the picture :+1:

The module relies on the input_polldev kernel thread for also fetching video data; that was intentional to keep both streams in sync. Therefore, if you completely disable the input device, the video stream also stops. However, if you just detach it using xinput float $SUR40, then the video stream is still usable.

[mkalten]

Wow, it's great to see that everything is working now! Many thanks for testing, and most of all thanks to @floe for implementing the V4L support on the SUR40!

@norpol, could you maybe post another test image that shows the fiducial and finger tracking feedback in the treshold mode? I am not sure how the visual artifacts from the screen content will affect the threshold and recognition performance ...

[floe]

BTW @norpol: how many FPS do you get?

[norpol]

Around ~35 FPS with default configuration. Screenshots soonish.

[mkalten]

OK, I finally also got hands on a SUR40 and already managed to test everything myself. On Ubuntu using a 4.1rc2-realtime kernel I actually get around 46fps ... good job @floe !

Here are a few tips to configure reactivision to work properly with the pixelsense image:

• background subtraction (empty table, fullscreen F1, spacebar)
• tresholder: gradient (G) and tilesize (cursor down) both at maximum
• finger size (F) at 20 and finger sensitivity (cursor down) at 100
• set the CPU governor to "performance" to increase CPU speed for lower latency
• turn off the reactivision display (N) to decrease CPU load

[mkalten]

good news @floe, I now managed to get the full 60fps from pixelsense just by reducing the POLL_INTERVAL to 4ms in sur40.c

reactivision now uses roughly 40% CPU for processing 60fps

[floe]

facepalm @mkalten thanks a lot for figuring that out! Didn't see the wood for the trees, apparently ;-) Will submit a kernel patch ASAP, maybe it will still get included into 4.1 release. Otherwise 4.2.

[mkalten]

I just created a pull request for your SUR40 driver, which also implements two further V4L functions for frame size and frame rate query

[nylki]

Oh nice it works! Just had to compile the recent kernel module.

@norpol In case you are reading this, could you elaborate? I am trying to get a raspi-cam working with reactivision 1.5.1 as well as 1.6.0 but I always get no V4Linux2 devices found.

[mkalten]

this seems to be the wrong thread, but you'll need to load the according V4L2 modules to access the raspicam via video4linux2: sudo modprobe bcm2835-v4l2

[mkalten]

@norpol btw, check out the latest version of @floe sur40 kernel driver ... we recently implemented more V4L2 controls, which now allow to set up the overall image brightness, contrast and gain ... this basically replaces the windows calibration and works much smoother!

https://github.com/floe/surface-2.0

[norpol]

Really nice, will check out!

[mkalten]

good @norpol , just one more remark: the new module works with kernels 4.9 and later .. and you'll need to load it with the option sur40_camera=true