Why is spherical projection model

[maqiang6]

I have found you project the plane image to spherical image. Then use Optical flow to obtain the displacement between two spherical image. But according to ideal rotating stereoscopic panoramic model, the cylindrical projection maybe more suitable. why not cylindrical projection?

[fbriggs]

It is equally correct to user either cylindrical or spherical as the coordinate system for optical flow. In either case, if you have a ring of cameras, then there are rays you want, but cannot get from novel view interpolation between real cameras (which can only generate cameras in between real cameras). So in practice, rays from cameras at the horizon are taken instead, which causes objects closer than infinity to appear vertically stretched. That would happen wether you use cylindrical or spherical. That is true of the "rotating stereoscopic panoramic model" you mentioned.

However, there are some reasons to prefer spherical to cylindrical. For example, if you are using fisheye lenses on side cameras, as the vertical FOV approaches 180 degrees, you would need an infinitely tall cylindrical image, whereas you don't with spherical.

There is also the issue of stitching the top and bottom cameras with the side cameras. That can't really be done in cylindrical, but can be done in spherical.

If you don't have fisheye side cameras, it is reasonable to do flow between side cameras in cylindrical, then project to spherical. There are some tradeoffs there, e.g., it costs time to do that additional projection, and it will also result in some degradation in image quality because it introduces an additional bilinear or bicubic interpolation.

[maqiang6]

Thank you for giving me an reason to choice spheical. But some issues that are still unclear to me.

1. you have said "if you have a ring of cameras, then there are rays you want, but cannot get from novel view interpolation between real cameras " .

   I don't know why I can't get from novel view? We have a ring of cameras. Can't we generate all the rays using Optical flow? Can you show me an example which rays we can't get from novel view.

2. you have said "which causes objects closer than infinity to appear vertically stretched" .

you mean this will cause the object looks thinner? So, the same object in right panorama and left panorama maybe not the same size?

[fbriggs]

1. with a ring of cameras, you can get novel views between real cameras, but the virtual cameras are on the same ring. to construct a perfect stereo panorama, you need rays that originate on a ring with diameter = IPD. for such rays, unless the direction is perfectly horizontal, you can't get them from cameras on the ring, and instead, you would need a camera above or below the ring.

2. objects close to the camera will look taller and thinner than they should. the amount of this effect depends on the radius of your ring of cameras (if it is 3.2cm, then this effect goes away), and the distance to the object (this effect goes away at infinity).

For more info on this issue, I suggest reading Google's paper on the JUMP system.

[maqiang6]

Thanks for you reply. I have understand why it looks taller. But I have another question. I use your palace dataset to generate stereo panorama without bundle adjustment rectification. I have found the building in left panorama is wider than right panorama.Have you found this problem? Can you give me a suggestion what may casue this problem?

[fbriggs]

You said that you didn't run bundle adjustment or rectification? That is the problem.

[maqiang6]

Yeah! After BA(bundle adjustment) the building is at the same width.

I wonder that why the width of the building is not same without BA. And the purpose of BA is to minimize vertical parallax. Why is the horizontal width same after BA?