Hotkey & auto capture support for CrossMgrVideo (without rfid)

[rkantos]

It is quite simple to implement a photoelectric sensor that would trigger a joystick button press: https://www.youtube.com/watch?v=4HW11sDh-5M

You can get a photoelectric sensor with a reflector for 20-30€ and a USB joystick encoder for less than 10€ from Aliexpress. This way one could use CrossMgrVideo without rfid-tags using the joystick button presses as a trigger for the capture.

Also what could be implemented is “auto capture” where CrossMgrVideo would check for movement in the image? Perhaps there would be a good ready-made function in OpenCV?

[esitarski]

The simplest way to do this is to add an Accelerator table to respond to a key code to trigger a capture. That would create a method for a photosensor or joystick button to trigger the capture. The down side is that CrossMgrVideo would need to have the keyboard focus - otherwise the key code would go to whatever application had the keyboard focus.

Movement checking from the images is an interesting idea. OpenCV has functions to do this (see here). Also here. You then have to threshold the subtracted images to get a binary image, then run "getContours" on bit bitmap, then check the area of the contours to see if there was a bit enough change.

I have two concerns:

1. The OpenCV functions take parameters that improve their performance under different conditions (lighting, how much the background is moving, etc.). How much parameter tuning is required in the real-world (a.k.a. what does it take to actually make it work)?
2. Can we process the frames fast enough? We don't want to drop frames.

More research required.

[rkantos]

The simplest way to do this is to add an Accelerator table to respond to a key code to trigger a capture. That would create a method for a photosensor or joystick button to trigger the capture. The down side is that CrossMgrVideo would need to have the keyboard focus - otherwise the key code would go to whatever application had the keyboard focus.

Movement checking from the images is an interesting idea. OpenCV has functions to do this (see here). Also here. You then have to threshold the subtracted images to get a binary image, then run "getContours" on bit bitmap, then check the area of the contours to see if there was a bit enough change.

I have two concerns:

1. The OpenCV functions take parameters that improve their performance under different conditions (lighting, how much the background is moving, etc.). How much parameter tuning is required in the real-world (a.k.a. what does it take to actually make it work)?
2. Can we process the frames fast enough? We don't want to drop frames.

More research required.

I see you've added code for supporting joystick functionality. Are you sure about requiring keyboard focus when using a joystick? All games I've been playing with a joystick on Windows or Linux usually work even when the window is not in focus. Since making this feature request, I've already built a joystick photosensor setup. I will test it soon with your latest commit!

[esitarski]

I don't have a joystick, but it looks like the joystick interface does not require keyboard focus.

Question: How should the photosensor joystick capture to work?

Currently, pressing Button 1 on the joystick starts capture, and releasing it stops capture.

I expect that you want the photosensor to trigger Auto Capture. An easy solution would be link Autocapture to Button 2. Or, do you have another suggestion?

[rkantos]

I don't have a joystick, but it looks like the joystick interface does not require keyboard focus.

Question: How should the photosensor joystick capture to work?

Currently, pressing Button 1 on the joystick starts capture, and releasing it stops capture.

I expect that you want the photosensor to trigger Auto Capture. An easy solution would be link Autocapture to Button 2. Or, do you have another suggestion?

That sounds logical, but the photosensor way will need to have some logic involved to work properly.

I think for a start an easy way to have it configurable and easy to understand, would be to have a slider for the number of frames (and the time that it equals to) that should be captured when the button of an actual joystick is pressed (or the photosensor triggers it) . Maybe have separate settings for Button 1 & Button 2? Both of the buttons could then be configured as either toggle/ on/off or as a "photosensor".

To be honest I don't know what the best way would be to allow the type of photosensor to work that I am currently using. The photosensor I use currently will just switch as quickly as possible. I have yet to test it with real cyclists, but the switching spec on it is in ≤ 20 ms, so it could trigger 50+ times per second. I imagine in reality it will trigger maybe ~25 times per second. For completeness, and also testing, maybe a switching limit could be set in milliseconds individually.

I guess it would work so that when the photosensor is triggered first, a timer would be created, and the time would be recorded (this would make <100ms TT timing possible). After creating the timer, CrossMgrVideo would capture frames either as long as there has been another time set by the user in ms from the number of frames to record. After that the capture could just be stopped after the number of frames configured has been recorded, and only record a time after the same amount of time has elapsed as is the amount of frames to be captured (1000/60fps*5fps = 83.333..ms). At 50km/h the times would then practically be accurate to 1/10, while there obviously would be duplicate times, every gap larger than that would get it's own time, more accurate than 1/50. Naturally the camera's "accuracy" remains at the framerate, and will obviously always be superior to a photocell, when the framerate is higher than the photocells triggering speed. The time recording functionality should again of course be configurable by the user, and perhaps enabled by default when in TT mode.

Going further it would be cool, if the joystick button could be configured to work in conjunction with rfid. For example the photosensor wouldn't record extra frames, when the capture was already triggered by rfid. The photosensor could be set (configurable, maybe by default) to either capture when it detects movement, only capture with rfid or with both the joystick button 1 / joystick button 2.

[esitarski]

Quick Update: I ordered a "Thrustmaster" joystick from Amazon. Button 1 is the main trigger. Holding it down does a capture. Button 2 is on the top of the control. It does an Auto Capture. Works well.

[mbuckaway]

Feature request: Space Invaders easter egg.

[esitarski]

CrossMgrVideo now supports "Space Invaders" mode. Upload your best results to High Score (http://highscore.com/) automatically through the embedded API. Revealing the key combination that enables "Space Invaders" mode would spoil the fun. Don't worry though, you will know when you find it.

[kimble4]

To be honest I don't know what the best way would be to allow the type of photosensor to work that I am currently using. The photosensor I use currently will just switch as quickly as possible. I have yet to test it with real cyclists, but the switching spec on it is in ≤ 20 ms, so it could trigger 50+ times per second. I imagine in reality it will trigger maybe ~25 times per second. For completeness, and also testing, maybe a switching limit could be set in milliseconds individually.

One neat way to do this could be to use an Arduino board that can emulate an HID, such as the Leonardo, with this joystick library: https://github.com/MHeironimus/ArduinoJoystickLibrary instead of an actual joystick.

You could then program it to generate joystick button events of a preferred duration, to wait before generating repeat events (the technical term for this is 'debouncing'), or to give up and stop generating events if the beam has been continuously broken for too long (fallen out of alignment, someone standing in front of it, etc). Obviously Arduinos are ideal for interfacing to homebrew sensor electronics, with plenty of tutorials out there for electronics newbies to get started.

(I did this the other day to try out the joystick feature, because I didn't have a joystick to hand, but did have a Leonardo. It grabs footage of me plugging a bit of wire into the Arduino's IO pin with split-second accuracy :) )

[kimble4]

Followup to the movement detection idea: I experimented with sticking a call to QRCodeDetector.detect() in the CamServer loop, on the basis that detecting (without decoding) a QR code was a relatively simple (so hopefully fast) thing for CV to do that would be robust against background movement. The idea that you'd position the QR code target across the line from the camera so it would be occluded by passing cyclists, and failure to detect a QR code (which requires all three 'eyes' to be visible) would trigger a capture.

On my few-years-old i7 desktop, with 1Mpix camera at 60fps, I could call detect() about 10 times per second in the main loop without affecting performance. Doable, and performance could likely be improved by doing the detection in a different thread.

At which point I abandoned the idea because the prospect of making a QR code target that, with our camera resolution and optics, would have to be somewhere between 1-1.5 metres wide, portable enough to carry to races, and somehow not flap about in the breeze causing false positives seemed impractical.

On the other hand, if you're working indoors, or you've got a solid object (wall, vehicle, podium, whatever) you can decorate, it might be a viable approach.

Meanwhile, I've gone down the optical beam rabbit-hole. Results seem promising, but I've yet to try aligning optics under real world conditions.