CalibrateProjector back to front when upside down

[elliotwoods]

when solving for a projector which is upside-down, we sometimes get a reversed frustum (needs checking tbh).

[mediadog]

Aha! I think I am experiencing a problem related to this. I have a three-projector setup pointing at a 3D city model. Two of the projectors cal just fine, but the third fails massively. One projector is on its left side, one is right-side-up, and the third is on its right side. They are typical business class projectors with a positive image shift. The one failing is the one on its right side.

I am calibrating them with twelve points, two planes of six each. I've tried shifting the points around to no avail. However if I reduce the number of points for that one projector to the four corners of one plane, it gets it sort of close. Adding any more points to that plane or points in another plane fails.

Here's the data:

World 0.938, 0.000, -0.480 0.938, 0.155, -0.480 0.938, 0.000, -1.195 0.938, 0.155, -1.195 2.275, 0.000, -0.496 2.275, 0.155, -0.496 2.275, 0.000, -1.215 2.275, 0.155, -1.215 3.395, 0.000, -0.494 3.395, 0.155, -0.494 3.395, 0.000, -1.208 3.395, 0.155, -1.208

Projective -0.769822376854056078, 0.512970762375633327 -0.878166391991698658, 0.575962888359885294 -0.833363852774217206, -0.086980406518952572 -0.941707867911859897, -0.042544100592077153 -0.396935848135262170, 0.474088994689617294 -0.510468168223158103, 0.551852530061649249 -0.476042238906183246, -0.225904901422205956 -0.591710919855948259, -0.174021851919672843 0.034364890435207227, 0.425929316974913041 -0.080266129524507110, 0.507416224134773830 -0.068790819752182139, -0.383324177501068175 -0.186473783800280790, -0.331441127998535090

Any ideas? Guess I can try flipping that projector...

[elliotwoods]

Hey! How about flipping the coordinates before you put in, then applying a 180 rotation to the projection matrix on the output

Sorry to suggest something hacky

[mediadog]

Good idea, but that does not appear to work with a lens shift; the shift is upside down as well and would require rotating 180 around the line from image to target, not the projector axis, unless I'm missing something. Looks like just flipping the projective Y and then rotating the result 180 around X might work. Or not, just tried it no improvement.

Hmmm, turns out that first projector on its side is not working so well after all; when decomposing the viewprojections only the right-side-up projector is giving a reasonably correct real-world position.

I'm wondering if this is not a weakness of the CV camera calibration; I assume it is designed with normal cameras in mind where the lens is coaxial to the line from target to image, not offset as in a view camera and most projectors. Would there be in benefit in seeding it with a more appropriate set of intrinsics to start with? I don't know anything about what is contained in "intrinsics", but do they cover such dramatic lens shifts? Normal cameras might have a slight coaxial error, but nothing like the 50-70% shifts of projectors.

Thanks a bunch Elliot!

[mediadog]

I managed to find a fix/work-around for my failing case. If I use something like a Projector helper and then an Instrinsics (Join) and feed that into the Intrinsics Guess pin of CalibrateCamera, it works, even if the projector definition is just the default. However, if I use an accurate projector definition with a V shift of over .5, the calibration fails with "OpenCV: Principal point must be within the image".

So two things, 1) the default Instrinsics Guess is not sufficient, and giving it a simple Perspective makes it work, and 2) a true guess with a shift of over .5 will not work. But hey at least I can get it to calibrate now!

One more thing: If I try a ProjectorExtrinsics, giving it accurate intrinsics and the same data set as CalibrateProjector, it says OK but the projector is shown facing ALMOST backwards; it is not exactly 180, so I cannot see how to easily correct it.