Can't find displays on Windows 10

[robert-virkus]

Hi there, I'm not sure if there is a forum somewhere dedicated to the RoomAliveToolkit. My problem is that nothing is displayed during the calibration process. When I select Setup -> Show Projector Server Connected Displays nothing is being displayed. In MainForm.ShowDisplayIndexes() two displays are being cycled through, the projector.Client.OpenDisplay() and DisplayName() methods are called without any error for both displays.

My environment is Windows 10 (build 10074) on a Surface Pro 3, I used VS2013 to build the project.

Any idea what's going on there?

Many thanks in advance, Robert

[thundercarrot]

Robert,

Offhand I don’t know what the problem is. Can I trouble you to run ProjectorServer in the debugger and set a breakpoint to verify that OpenDisplay is called in the server, and that ProjectorForm.OnLoad is not failing silently somewhere? projectorForm.Show should result in a white window appearing.

Andy

[robert-virkus]

Andy, thanks for the quick response!

I don't encounter any issues within the ProjectorServer. Within debugging I have managed once to view the display0 (on my mainscreen) but no luck with display1. In ProjectForm both initialization and load is called seemingly correctly, for the extended display 1 I get the following bounds (which seem to be fine IMHO): X = 2160, Y = 0, Width = 1024, Height = 768.

Could this be a Windows 10 preview problem? PowerPoint has no problem to use the extended display for the presentation and the main screen for the presentation notes.

Thanks again, Robert

[robert-virkus]

I've managed to progress one step further. In ProjectForm I now use the following hardcoded bounds values: X = 1440, Y = 0, Width = 690, Height = 490. With these values I almost fit in perfectly (note that Windows claims to run the projector in 1024x768 mode). During Calibration -> Acquire the images are shown. I noticed that "acquire depth image XX" messages are continuing for much longer than images are shown, but I'm not sure if this is normal. When calling Calibration -> Solve I always end up with an infinite error value at the end. Of course I tried moving in chairs into the image, but I'm always having several NaN error messages and an infinite error - like this example:

decoding Gray code images for projector 0, camera 0 elapsed time 4337 projecting depth points to color camera 0 rejected 23.86175% pixels for high variance elapsed time 4669 elapsed time 4932 calibrating projector 0 RANSAC iteration 0 error = NaN error with inliers = NaN camera matrix = 1000, 0, 512 0, 1000, 0 0, 0, 1 RANSAC iteration 1 error = NaN error with inliers = NaN camera matrix = 1000, 0, 512 0, 1000, 0 0, 0, 1 RANSAC iteration 2 error = NaN error with inliers = NaN camera matrix = 1000, 0, 512 0, 1000, 0 0, 0, 1 RANSAC iteration 3 error = NaN error with inliers = NaN camera matrix = 1000, 0, 512 0, 1000, 0 0, 0, 1 final calibration: camera matrix = distortion = error = ∞ elapsed time 5073

Any idea what's going on?

Many thanks in advance, Robert

[thundercarrot]

Robert, I'm wondering if you would consider zipping up the entire directory that includes calibration.xml and putting it somewhere where I can download it. CalibrateEnsemble.exe can run the calibration routines offline and maybe I can get an idea as to what's going on with your setup. Andy

[robert-virkus]

Hi Andy,

Sure: https://onedrive.live.com/redir?resid=BFB8A4335206A652!271681&authkey=!AMKd_bSjHLmmb-M&ithint=file%2czip

I also tried to change ProjectorServer.Size(int screenIndex) and returned a fixed size for screenIndex=1, but that also resulted into an indefinite error.

Thanks for all the help, Yours Robert

[thundercarrot]

Robert can you double check that link? It just brings me to my own set of shared files.

[robert-virkus]

Andy, maybe this shortened version works better? http://1drv.ms/1e0CZZh Thanks, Robert

[thundercarrot]

This link fails also. Hmm. Find another way?

[robert-virkus]

Sorry for the late feedback, Andy. Here's it on Dropbox: https://www.dropbox.com/s/zxvi2w3pr454rmc/CalibrationResult.zip?dl=0

[thundercarrot]

I took at look at your images. It seems the projector is completely blowing out the color camera image. The Acquire phase of calibration attempts to mitigate this by putting up a gray image for some time before acquiring the Gray code images, but in this case AGC is not doing the right thing for us, perhaps because the projected image is rather small compared to color camera frame (I don't know how the Kinect color camera does AGC, if it looking at a spot in the center of the image or the whole image). Also, there's very little light in the scene. A small projected image is also tricky because it limits the number of bits that can be recovered by the Gray code. In another case I'm chasing that has prevented calibration from succeeding. My suggestion would be to rearrange your setup so that the projected image is bigger. You might consider adding a bit of light too, but that is probably not necessary once the projected image is larger.

I'm still stumped as to why your projector image dimensions are incorrectly reported. Do you by any chance have another projector or even another LCD panel that you could check?

[robert-virkus]

Hi Andy, many thanks for your analysis! I'll do as you suggested and report back my findings, however that won't be before Sunday as I'm traveling right now. Thanks again for all your help! Yours Robert

PS: The funny thing is that the reported dimensions are correct (both the projector and Windows claim that it's 1024x768), but that they don't work when arranging the forms. I assume this is a Windows 10 bug.

PPS: The result that you have is with lights out, I tried that in the hope to improve contrast - but it was the same result as with lights on.

Sent from my C64

---

Von: Andy Wilsonmailto:notifications@github.com Gesendet: ‎14.‎05.‎2015 17:55 An: Kinect/RoomAliveToolkitmailto:RoomAliveToolkit@noreply.github.com Cc: Robert Virkusmailto:robert.virkus@enough.de Betreff: Re: [RoomAliveToolkit] Can't find displays on Windows 10 (#2)

I took at look at your images. It seems the projector is completely blowing out the color camera image. The Acquire phase of calibration attempts to mitigate this by putting up a gray image for some time before acquiring the Gray code images, but in this case AGC is not doing the right thing for us, perhaps because the projected image is rather small compared to color camera frame (I don't know how the Kinect color camera does AGC, if it looking at a spot in the center of the image or the whole image). Also, there's very little light in the scene. A small projected image is also tricky because it limits the number of bits that can be recovered by the Gray code. In another case I'm chasing that has prevented calibration from succeeding. My suggestion would be to rearrange your setup so that the projected image is bigger. You might consider adding a bit of light too, but that is probably not necessary once the projected image is larger.

I'm still stumped as to why your projector image dimensions are incorrectly reported. Do you by any chance have another projector or even another LCD panel that you could check?

---

Reply to this email directly or view it on GitHub: https://github.com/Kinect/RoomAliveToolkit/issues/2#issuecomment-102082122

[thundercarrot]

I'm not seeing this issue on the same build of Windows 10. Do you by any chance have any dpi scaling running on your setup?

[kjancsi]

Hello,

First off, thanks for making the code available. I'm also having trouble when calling Calibration->Solve, getting NaN's for errors, similar to Robert's. I'm running the code on Win8.1. I've tried different playing with both the lights and the size of the projected image, but neither helped. Any ideas what else to try?

Btw, before setting the scaling on the display to 100%, I had issues with the calibration stripes only covering the right half of the projected surface. With scaling set to 100%, things work fine, i.e. the patterns cover the entire projected screen.

Thanks, kj

[thundercarrot]

KJ, Can you zip up the directory with calibration.xml and post it somewhere I can download it? I would be happy to take a look. Andy

[kjancsi]

Hi Andy, Thanks a lot, here's the zip file: https://drive.google.com/file/d/0B9ldkR9liykmbXBHekNHb3prX2s/view?usp=sharing I noticed that KinectServer.exe lists "color error" and "color projection error" as NaN. Let me know if you have trouble accessing it. I have tried projecting into corners, having more furniture in front of the wall projected onto, etc to no avail. Cheers, kj

[thundercarrot]

kj,

The error message from KinectServer.exe leads me to believe that you have stumbled upon a rare failure mode of KinectServer.exe (or the camera) where the SDK's coordinate mapper functions return invalid values (step into Kinect2Calibration in the debugger and look at the values returned by CoordinateMapper). This corrupts the overall calibration.

I'm not sure what the issue is; we have observed this in a few cameras here at the lab but have yet to resolve the issue. In some cases the problem seems to go away after time?

Do you have another Kinect v2 sensor you can try? Andy

[ChimeraScorn]

This is a known (and expected) condition on first connection. Technically, the CoordinateMapper will not operate correctly until it has loaded calibration data from the device. When this happens, the CoordinateMapperChanged event will fire, after which time CoordinateMapping will operate correctly.

Most code doesn’t tend to care about this, as most code doesn’t try to do coordinate mapping until after it’s started to receive depth frames, and I’ve never seen the coordinate mapping not be available by the time the first depth frame has arrived.

-C

From: Andy Wilson [mailto:notifications@github.com] Sent: Tuesday, May 19, 2015 10:46 PM To: Kinect/RoomAliveToolkit Subject: Re: [RoomAliveToolkit] Can't find displays on Windows 10 (#2)

kj,

The error message from KinectServer.exe leads me to believe that you have stumbled upon a rare failure mode of KinectServer.exe (or the camera) where the SDK's coordinate mapper functions return invalid values (step into Kinect2Calibration in the debugger and look at the values returned by CoordinateMapper). This corrupts the overall calibration.

I'm not sure what the issue is; we have observed this in a few cameras here at the lab but have yet to resolve the issue. In some cases the problem seems to go away after time?

Do you have another Kinect v2 sensor you can try? Andy

— Reply to this email directly or view it on GitHubhttps://github.com/Kinect/RoomAliveToolkit/issues/2#issuecomment-103768307.

[kjancsi]

Hi, thanks. Unfortunately I do not have another Kinect v2 sensor to try. Btw, this is the full output I get when I launch KinectServer.exe.

Thanks, kj

rms error = NaN 1000 1000 960 540 0 0 0 0 0 0.0537956021675102 -4.44476167224205E-05 -0.00844434348715794

rms error = 0.0715197290976816 rms error = 0.0647115475565153 rms error = 0.0646594594070298 rms error = 0.0646588116065869 rms error = 0.0646588035503489 rms error = 0.0646588034501673 rms error = 0.0646588034489216 366.676414491692 366.702198735052 256.51617593772 211.091534613167 0.0595045679053993 -0.166649097853368

FakeCalibration : n = 2395 color error = NaN depth error = 0.0646588034489216 depth reprojection error = 0.00836152172689252 color reprojection error = NaN depth camera matrix = 366.676414491692, 0, 256.51617593772 0, 366.702198735052, 211.091534613167 0, 0, 1

depth lens distortion = 0.0595045679053993 -0.166649097853368

color camera matrix = 1000, 0, 960 0, 1000, 540 0, 0, 1

color lens distortion = 0 0

2549 ms

[thundercarrot]

@ChimeraScorn KinectServer performs these calculations in a CoordinateMapperChanged callback.

There's something more sinister going on here. kj, If you don't have another unit, might I suggest unplugging the Kinect USB and power for a while, and maybe trying a different PC. Also double check that you are not still running the pre-release version of the Kinect SDK, etc.

[kjancsi]

Hi, I tried it on another PC (this time on the most recent public build of Win10), same result. Kinect SDK is the latest, the Kinect Configuration Verifier reports every aspect to be ok, and the Kinect SDK samples work fine.

[thundercarrot]

kj, I don't have a camera that is exhibiting this behavior, so it is very difficult for me to debug this. If you have a bit of coding skills, might I suggest changing KinectServer so that code in the CoordinateMapperChanged event handler is called again when the server gets the request for the camera calibration structure. Running that process later might work around whatever bug/problem your camera is exhibiting.

[thundercarrot]

btw I've opened a forum for the RoomAlive Toolkit

Andy