Ultra Short Throw projectors invalid?!

[NPatch]

Hi, I'm trying to use a Dell S500 projector with RoomAlive but I cannot achieve a proper calibration. Projection View after solve is either a black picture or just wrong(mostly the former). The projector is mounted on the ceiling and apart from being very close to the wall(since it's an ultra short throw) every other step in the tutorial is covered. I use 60" tv screen boxes which cover a large area of the projection(almost half) to avoid the flat projection area issue. Tried with the projector both mounted on the ceiling and on the floor, but no dice. I only got a correct calibration once but haven't managed to make it work since.

For what it's worth, I've tried with a Dell SP890 and even though it can't do a big projection area(even if placed at 4-5 times the distance from the wall as the S500), I always get correct calibration results(that means ProjectionView is always covering the grid precisely).

At the end i concluded it might be due to the ultra short throw. Is this conclusion correct?

[thundercarrot]

I've never calibrated an ultra-short throw projector but I've often wondered whether it would work. Can you zip up your calibration directory and put it somewhere where I can download it? I will take a look.

My feeling is that the math is no different for an ultra short throw but there might be some numerical stability problems, or sensitivity to the initial guess at the intrinsics.

Thanks for bringing this to my attention!

[NPatch]

Sure. Here it is. DellS500.zip

Also in Unity, out of all the times I tried to build the scene using the ultra short throw, almost all of them had different place and rotation for the Projector object and almost always wrong frustum. Once it the projector was towards the right and the frustum going towards the left. Sometimes the projector was lower than the Kinect and Scene objects , or even behind the Scene object.

I imagine that the nature of the projector somehow messes up the calculations but it's weird how it's almost never the same. Unless I'm doing something totally wrong here.

I'm wondering whether one box like this is enough to calibrate the first time. For what it's worth I tried with this projector at least once where I used two boxes like that , covering half of the projected area and placed in the middle so it doesn't affect the sides of the projection. The only issue I see is that regardless of this projector's position(same distance from wall but mounted on ceiling or placed on the floor), I always have shadows from the boxes. The worst case being when the projector is on the floor. The light comes from down below so the shadow the box casts is quite long and I'm wondering whether it might affect the gray codes.

If mounted on the ceiling, the boxes being a bit bulky, end up cutting the light coming from above at an angle so the light doens't go all the way to the floor as it would without the box. Here's an example: Forgot to add it, the Kinect is to the right, on the floor, at a distance and tilt to be able to see the whole projection area as well as some floor in front of the box.

[thundercarrot]

I took a look at your calibration (sorry it took me a little while to get to). It is clearly not calibrating correctly, but I don't see anything obviously wrong in your setup. I think there are some things you can try to make things easier for the calibration however:

1. Get the projection off the floor. The depth data for such oblique surfaces may not be reliable enough to use.
2. Get some more variation in the geometry; the box there is not so far from the wall.
3. Take the projector down from the ceiling, and set up your camera so that it is close to the projector. This will make it easier to toss more geometry in the scene.

These might not match your intending configuration, but I think it is always best to walk before trying to run :-). Also, if you get good intrinsics from the projector in an easier setup you can lock the intrinsics (look in the configuration file) which will make it easier to calibrate when the projector and camera are not at the same place. LMK how it goes!

[NPatch]

How do I know when I get good intrinsics? Is it the error numbers? I remember in Channel 9 you mentioned they had to be under 1 to be correct but so far I've never gone above it. 0.6-0.8 is the range I usually get.

[rolandsmeenk]

Note that an ultra short throw projector like the Dell S500 will have a very small depth of field. This limits it's usability to (fairly) planar surfaces (See Guide to Projectors for Interactive Installations http://www.ucl.ac.uk/slade/know/3672). I can imagine this makes it hard to properly project crisp gray patterns when depth of the scene varies and that impacts the accuracy of the calculations done in RoomAlive Toolkit.

On Thu, Jul 19, 2018 at 10:39 AM, NPatch notifications@github.com wrote:

How do I know when I get good intrinsics? Is it the error numbers? I remember in Channel 9 you mentioned they had to be under 1 to be correct but so far I've never gone above it. 0.6-0.8 is the range I usually get.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Microsoft/RoomAliveToolkit/issues/81#issuecomment-406200616, or mute the thread https://github.com/notifications/unsubscribe-auth/AAdUPJ2gbleD9CAxUmdGIrtCiSks2iDdks5uIEXBgaJpZM4VK_jZ .

[NPatch]

Well the intended setup will be projections on walls so basically planar surfaces. Or at least that's how it is right now. At the moment I'm short on short throw projectors and using a Dell 7700FHD which can be calibrated easily so I can familiarize with RAK(will change later since the projection size is small at 3-4 meters).

Thanks for the link though, I'm kind of new to using projectors so I don't have a clear list of things to check.

[thundercarrot]

In a good calibration, when you select the project under the View menu, you see the whole projection (grid lines and projector name) rendered right up to the edge of the window.

The thing that puzzles me here is that the calibration error is very low (though the number of points that wound up in the calculation is quite low-- only 526 out of 68339). Meanwhile the focal length calculated is negative, suggesting that calibration came up with a mirror image solution (focal length should be positive).

I noticed that the text in the projected image is upside down. One quick thing to try is to flip the image on the projector if that is possible (or turn the Kinect upside down). Just a hunch I have that the optimization is having trouble getting away from some dual solution.

[NPatch]

The Projector is hung upside down and is set to Front, so it renders upside down. I know the Projection View will render something wrong if the calibration is faulty somehow(with the S500 I got it showing something correct only once, but every single other time I mostly got a black/blank view). I was asking whether there was a tell when it came to the solve logs. The error values there almost never go beyond the 0-1 range even if the end result is wrong. I'll try using the Kinect upside down and reupload the calibration data.

[thundercarrot]

The log dumps the camera matrix; from there you can see that the focal length is negative. It also shows the number of inlier points used in that round of optimization, out of the total number of possible points that could be used.

Your projector may have a Ceiling mount option. That would flip the image.

[NPatch]

Ok I'll check them out next time I attempt the calibration.

Indeed it does have Front Ceiling. I left it to Front since the tutorial says to set it to Front. I figured there wouldn't be a problem since the gray codes seem like they wouldn't make a difference whether upside down. They seem symmetrical enough to not meddle. I'll post two calibrations then. I'll improve the clutter and post one as-is and another one with Front Ceiling.

[NPatch]

I just tried calibrating again with the S500. Tried both Front and FrontCeiling. Used more stuff to add some complexity to the flat surface(more than the previous time). But it still fails. Here's a link.

UPDATE: After reading your initial comment again, I tried to move some clutter outside the projector frustum and nearer the Kinect in case that provides more points. Here's a link of the same calibrations in both front and front ceiling.

I'm concerned about the voids in the calibration attempts. What do you think it is? Is it due to the Kinect being on the floor looking slightly upwards?

[thundercarrot]

I took a look at these new calibration sets. The objects you added don't help much, as they don't return any projected light (are they black?). See grayCode*.tiff, and mask.tiff in the same directory.

The tutorial says to use desktop front projection because going into the calibration, there is a initial value for the projector's principal point that reflects typical lens shift you have in most regular (non-ultra short throw) projectors, where the principal point is usually near the bottom of the image. I fiddled with that but still the calibration fails in the same way (negative focal length).

The more I think about this, the more I believe the problem is that there isn't enough variation in depth in the scene, and that will be hard to achieve with your current configuration. If you take the projector down and then put some things at a vareity of depths that might help. Getting it down on the floor will make it easier for you to place them; then place the camera so you can actually image them. If you can get a successful calibration this way, you can always set 'lockintrinsics' to true in the projector part of the xml file, mount the projector on the ceiling and then re-calibrate to compute extrinsics.

[NPatch]

In the mask.tiff, I can see two things that are black. I think I've removed them since. The one with the holes in the middle is made of wood but I think it's also varnished so I took it out in subsequent calibration attempts(I've come across two BenQ short throw projectors so I've switched out my setup). The wood is dark brown color. I don't know if the dark color or the varnish can absorb the IR but it's probably one of the two(though I'd expect the varnish to reflect since it's the top most layer and it's usually reflective). The other one is a cardboard box. The weird part is that I'd assume only the lid would be absorbent but in the "differentlayout" calibration, I can see the sides are black as well. Perhaps some weird sort of pigment or finish was applied. Honestly, I think the toughest part in this is to find proper clutter and position them well.

I guess your idea to first calibrate with the projector below to get the intrinsics and then locking them to redo it from above, is the most practical.

Could you perhaps comment on the other issues I've created?!

At any rate, thanks a lot for trying. When I find the time and a spare ultra-short S500(the one previously used is part of a different project and I was only borrowing it for quick tests), I'll try it again and post my findings.