Fundamentals of panorama viewer

[exotfboy]

Hi:

This is a wonderful tool for panorama viewing, however I want to know how it works, how do you generate the tiles, how do you load the correct tiles or adjust the angle of the image according to user interaction and etc. In a word, how to build a same wheel like yours. :)

You may not spare so many time to tell me this, but can you give me some keywords or some tutorials?

[mpetroff]

For the multiresolution mode, tiles are created by projecting to a cube map and then creating an image pyramid for each cube face. These are displayed using WebGL by creating a corresponding cube geometry, with the cube faces again tiled; the geometry is then observed with a pinhole camera model, and the tiles are mapped as textures onto the tiled cube faces, where each tile consists of a pair of triangles.

For equirectangular mode, the projection is done entirely with a fragment shader, which is used to texture a pair of triangles. Texture coordinates are calculated using a pinhole camera model and mapped from the equirectangular image. For details of the math involved, see: doc/pannellum-equirectangular-projection-reference.pdf.

To get a better sense of implementation details, I'd recommend looking at the code, particularly src/js/libpannellum.js.

As for tutorials, I found Opera's Raw WebGL 101 article series helpful when I wrote most of the WebGL-related code in 2013: https://dev.opera.com/articles/raw-webgl-part-1-getting-started/ https://dev.opera.com/articles/raw-webgl-part-2-simple-shader/ https://dev.opera.com/articles/raw-webgl-part-3-advanced-shader/ https://dev.opera.com/articles/raw-webgl-part-4-textures/

At the time, there was very little in the way of tutorials or examples on how to use WebGL without a framework library such as Three.js. I don't know if that has changed since then or if there are now better resources available.

[exotfboy]

Thanks for your reply.

I agree that we should use a framework like three.js instead of writing raw webgl.

While I am interested in the projection you mentioned multiresolution and equirectangular. I am familiar with Openlayers, LeafletJs and etc which use the map projections to display the 3D world in a 2D flat screen like EPSG3857. Is the panorama viewer a reverse operation to project the 2D image back to a 3D surface like a cube? How do we calculate the tiles? For the EPSG3857 projection, for a given zoom level, we know the tile size 256 and the tile numbermath.pow(2,n), then we can render each tile accordingly. How about that in a cube?

[mpetroff]

I never said anything about whether or not a framework should be used, so I'm not sure what you're agreeing with. Using a framework has a significant overhead but makes development easier. Thus, whether or not you should use one depends on your priorities. For Pannellum, runtime efficiency is a high priority, so it was worth the additional development effort to write raw WebGL.

Equirectangular projection is EPSG:4087, except it's unitless and uses lat/lon on a sphere instead of on the WGS84 spheroid. In this projection, the image x coordinate directly matches the longitude and the image y coordinate directly matches the colatitude, when both the image coordinates and lat/lon are normalized between zero and one. The way the rendering for the equirectangular mode works is by calculating lat/lon for each pixel on the screen using a pinhole camera model observing a unit sphere from inside at its origin. This lat/lon is then projected to the equirectangular image (which is more of a direct lookup than a projection) and used to retrieve the color.

Pannellum's multiresolution projection is a Gnomonic projection onto each cube face. The tiles are then created by subdividing the projected image for each face. For rendering, a cube is created, the Gnomonic projection is displayed on each face (using the tiles), and this geometry is observed from inside at the origin using a pinhole camera model. Since a Gnomonic projection is effectively the same as a pinhole camera model, viewing the cube with Gnomonic projections on the faces with a pinhole camera model is equivalent to viewing a sphere with a pinhole camera model. For a given cube face, the tiling works pretty much the same as the tilings used for slippy maps; the only difference is that a Gnomonic projection is used to map the sphere to the plane instead of using EPSG:3857 to map the WGS84 spheroid to the plane.