Panorama integrated hotspots/interactions possible?

[ician-42]

Hello,

I have used Pannellum in multiple panorama displays already. I really appreciate it and thank you making it available. When it comes to the hotspots I would like to have more integrated hotspots with the panorama. E.g. define areas inside the panorama where a user could click to create custom events. Ideally I would like to define an area with top and bottom, left and right pitch and yaw values to define a projected rectangle or a circle using a center pitch and yaw and a circumference in degrees.

As far as I understand pannellum right now, this is not directly supported by the library, and as stated in a similar request https://github.com/mpetroff/pannellum/issues/246 you think it is possible, but do not intend to add such a feature yourself.

Could you point me into the direction how I would proceed with this? Would it be best to use a third party library like three.js to add my own objects to the renderings? I am thinking about overlaying a canvas managed by three.js syncing the camera to the rotation provided by pannellum and using ray casters to have interactable elements. Alternatively I could add a rendering callback to pannellum, which allows my own elements to be added to the pannellum canvas.

Your help would be much appreciated! Thanks Ludwig

[mpetroff]

If you're okay with drawing the areas in question on the panorama, you could use Pannellum's .on() method of add event listeners for mousedown / mouseup and touchstart / touchend, feed the events to the .mouseEventToCoords() method to convert them to panorama coordinates, and then implement logic using that information. By drawing an equirectangular image to a <canvas> and using the .setUpdate() method to trigger updates, you could also add hover interactivity; this would probably be easiest to implement by having the hover effects drawn as separate equirectangular images that could be overlaid on the <canvas>.

Your idea of using Three.js could also work. Overlaying a separate <canvas> is definitely the approach to take for that. While it is possible to get Three.js and Pannellum to work together with a single <canvas>, it requires changes to libpannellum.js because WebGL uses shared state; the two libraries end of overwriting the shared state, so values have to be reset between render passes of the two libraries.

[ician-42]

Hello Matthew,

thank you for your response. I tried the first approach, and was semi successful. I made it work to render rectangles on the source image by transforming pitch and yaw back to source Image pixel sizes. So I can now draw my elements on there.

In the process I did some more research and came across a marzipano demo, that did what I need by overlaying html content and transforming it using css transformations. Like here: https://www.marzipano.net/demos/embedded-hotspots/

I tried to alter the renderHotspot method use the same logic as marzipano in a local fork, but was not successful. https://github.com/mpetroff/pannellum/blob/74b83191928fb0b39fb2888adb7be1f5daad1d0a/src/js/pannellum.js#L2041-L2078

This is the code I tried to adapt - translating rad and degree respectively. For the radius used, I tried to guess/calculate based on window width and hfov. https://github.com/google/marzipano/blob/eb116412d1b7c6df08906e9affcc9d3d65b5906f/src/views/Rectilinear.js#L832

I would love to have actual html being integrated in the panorama. Do you think that it is possible to use the code from marzipano and integrate it into pannellum? If so, is the renderHotspot, the correct location to do so?

My code example is not working close enough that I think it is just a single line/value that is wrong. So before I waste more time on it, I wanted to make sure it is a solution that might work, and the location where I try to alter pannellum is the correct one.

Thank you Ludwig

[mpetroff]

Using a CSS 3D transform with the perspective property in Pannellum's renderHotSpot function is the correct approach to take, so you're on the right path. The difficult part is figuring out the exact transform parameters to use and in which order (and getting it to work reliably across different browsers, although that part is probably less difficult now than it was when I wrote the multires / cubemap CSS 3D transform fallback renderer).

[ru13ab]

Hello, @ician-42 , Did you end up being successful? I need the same, I would be grateful if you leave a comment. Thank you in advance.

[ician-42]

@ru13ab: unfortunately i was not successful. In the end decided to use marzipano in projects, where I wanted to use this specific feature

[dbwodlf3]

@ru13ab not beautiful but It can be solved. Just you apply perspective transformation with distortion. (Not best performance, but it works and can be tolerated.).

create 4 hotspot. then you can get 4 before and after points. and you can calculate transformation matrix from 8points(before and after x y position).

transformation_matrix
[
    a1 a2 0 a3
    a4 a5 0 a6
    0  0  1 0
    a7 a8 0 1
]

`x = a1x + a2y + 0 + a3 + a4(0) + a5(0) + 0 + a6(0) + ( a7(-`x*x) + a8(-`x*y) )
`y = a1(0) + a2(0) + 0 + a3(0) + a4x + a5y + 0 + a6 + ( a7(-`y*x) + a8(-`y*y) )
...
`x4 = ...
`y4 = ...

And you can get a1~a8s value using by linear solvers (like numeric.js).

The idea is very simple. and it works.

// you get transformation matrix.

hotspot_element.style.position = "absolute";
hotspot_inner_img_element.style.position = "absolute";

hotspot_element.style.transform = `matrix3d(
                    ${solve[0]}, ${solve[3]}, 0, ${solve[6]},
                    ${solve[1]}, ${solve[4]}, 0, ${solve[7]},
                              0,           0, 1,           0,
                    ${solve[2]},  ${solve[5]}, 0,          1
                )`

[ChanderShekharKestone]

@ru13ab not beautiful but It can be solved. Just you apply perspective transformation with distortion. (Not best performance, but it works and can be tolerated.).

create 4 hotspot. then you can get 4 before and after points. and you can calculate transformation matrix from 8points(before and after x y position).

transformation_matrix
[
    a1 a2 0 a3
    a4 a5 0 a6
    0  0  1 0
    a7 a8 0 1
]

`x = a1x + a2y + 0 + a3 + a4(0) + a5(0) + 0 + a6(0) + ( a7(-`x*x) + a8(-`x*y) )
`y = a1(0) + a2(0) + 0 + a3(0) + a4x + a5y + 0 + a6 + ( a7(-`y*x) + a8(-`y*y) )
...
`x4 = ...
`y4 = ...

And you can get a1~a8s value using by linear solvers (like numeric.js).

The idea is very simple. and it works.

// you get transformation matrix.

hotspot_element.style.position = "absolute";
hotspot_inner_img_element.style.position = "absolute";

hotspot_element.style.transform = `matrix3d(
                    ${solve[0]}, ${solve[3]}, 0, ${solve[6]},
                    ${solve[1]}, ${solve[4]}, 0, ${solve[7]},
                              0,           0, 1,           0,
                    ${solve[2]},  ${solve[5]}, 0,          1
                )`

Hi @dbwodlf3, Please help me to solve this. Is there any online demo or repo wherer i can check this.

[rifkytech86]

Hi @dbwodlf3, +1 for @ChanderShekharKestone could you please help us what best way resolve it, i really appreciate your help

[ChanderShekharKestone]

I am also wating for answer from @dbwodlf3. As i got anything i will add on demo.

[ChanderShekharKestone]

Hi @dbwodlf3, +1 for @ChanderShekharKestone could you please help us what best way resolve it, i really appreciate your help

Hi @rifkytech86 did you get any solution. if get then please share...

[dbwodlf3]

@ru13ab not beautiful but It can be solved. Just you apply perspective transformation with distortion. (Not best performance, but it works and can be tolerated.). create 4 hotspot. then you can get 4 before and after points. and you can calculate transformation matrix from 8points(before and after x y position).

transformation_matrix
[
    a1 a2 0 a3
    a4 a5 0 a6
    0  0  1 0
    a7 a8 0 1
]

`x = a1x + a2y + 0 + a3 + a4(0) + a5(0) + 0 + a6(0) + ( a7(-`x*x) + a8(-`x*y) )
`y = a1(0) + a2(0) + 0 + a3(0) + a4x + a5y + 0 + a6 + ( a7(-`y*x) + a8(-`y*y) )
...
`x4 = ...
`y4 = ...

And you can get a1~a8s value using by linear solvers (like numeric.js). The idea is very simple. and it works.

// you get transformation matrix.

hotspot_element.style.position = "absolute";
hotspot_inner_img_element.style.position = "absolute";

hotspot_element.style.transform = `matrix3d(
                    ${solve[0]}, ${solve[3]}, 0, ${solve[6]},
                    ${solve[1]}, ${solve[4]}, 0, ${solve[7]},
                              0,           0, 1,           0,
                    ${solve[2]},  ${solve[5]}, 0,          1
                )`

Hi @dbwodlf3, Please help me to solve this. Is there any online demo or repo wherer i can check this. If not i also attached a link of online demo please guide me how to achive this when i rotate sceen image will matrix3d css. https://codesandbox.io/s/pannellum-react-57vogl

I think you solved the problems. you need to get just transition values. it is easy. you can get _x, _y value from hotspot element getBoundingRect and canvas left and top. and you can also know absolute value of (x, y).

left top = (0, 0) right top = (x, 0) right bottom = (x, y) left_bottom = (0, y)

and i leave some my code. it doesn't works directly, but i think it is enough to understand a logic.

  /** returns [yaw, pitch]*/
  static getCoords(inputX: number, inputY: number){

    // var pos = mousePosition(event);
    const container = PIManager.viewer.getContainer();
    const renderer = PIManager.viewer.getRenderer()!;
    const container_rect = container.getBoundingClientRect();
    const pos = {x: inputX - container_rect.left, y: inputY - container_rect.top};

    const config = PIManager.viewer.getConfig()!;

    const canvas = renderer.getCanvas();
    const canvasWidth = canvas.clientWidth;
    const canvasHeight = canvas.clientHeight;
    const x = pos.x / canvasWidth * 2 - 1;
    const y = (1 - pos.y / canvasHeight * 2) * canvasHeight / canvasWidth;
    const focal = 1 / Math.tan(config.hfov * Math.PI / 360);
    const s = Math.sin(config.pitch * Math.PI / 180);
    const c = Math.cos(config.pitch * Math.PI / 180);
    const a = focal * c - y * s;
    const root = Math.sqrt(x*x + a*a);
    const pitch = Math.atan((y * c + focal * s) / root) * 180 / Math.PI;

    let yaw = Math.atan2(x / root, a / root) * 180 / Math.PI + config.yaw;
    if (yaw < -180) yaw += 360;
    if (yaw > 180) yaw -= 360;

    return [yaw, pitch];
  }

  setRelativeSize(){
    this.initBaseXY();

    if(this.relativeSizeEvent) {
      PIManager.viewer.off("animatefinished", this.relativeSizeEvent);
      PIManager.viewer.off("zoomchange", this.relativeSizeEvent);
      this.relativeInit = false;
    }

    // Check Init
    if(!this.relativeInit) {

      const rect = this.element!.getBoundingClientRect();
      const left_top = PIManager.getCoords(rect.x, rect.y);
      const right_top = PIManager.getCoords(rect.x+rect.width, rect.y);
      const right_bottom = PIManager.getCoords(rect.x+rect.width, rect.y+rect.height);
      const left_bottom = PIManager.getCoords(rect.x, rect.y+rect.height);

      /** I think you need to add heres logic for yours one. */
      const left_top_hotspot = PIManager.addHelperHotspot(this.hotspotId, 1, left_top[0], left_top[1], 0, 0);
      const right_top_hotspot = PIManager.addHelperHotspot(this.hotspotId, 2, right_top[0], right_top[1], rect.width, 0);
      const right_bottom_hotspot = PIManager.addHelperHotspot(this.hotspotId, 3, right_bottom[0], right_bottom[1], rect.width, rect.height);
      const left_bottom_hotspot = PIManager.addHelperHotspot(this.hotspotId, 4, left_bottom[0], left_bottom[1], 0, rect.height);

      this.helpers.push(left_top_hotspot, right_top_hotspot, right_bottom_hotspot, left_bottom_hotspot);

      const _this = this;

      this.relativeSizeEvent = function(){
        const canvas = PIManager.viewer.getRenderer().getCanvas() as HTMLElement;
        const canvas_rect = canvas.getBoundingClientRect();

        const equations = [];
        const answers = [];
        const row1 = [];
        const row2 = [];
        const row3 = [];
        const row4 = [];

        for(const item of _this.helpers) {
          const current_xy = item.getXYOffset(canvas_rect.left, canvas_rect.top);

          const x = item.baseXY![0];
          const y = item.baseXY![1];
          const _x = current_xy[0];
          const _y = current_xy[1];

          const equation_x = [x, y, 1, 0, 0, 0, -x*_x, -y*_x];
          const equation_y = [0, 0, 0, x, y, 1, -x*_y, -y*_y];

          equations.push(equation_x);
          equations.push(equation_y);
          answers.push(_x);
          answers.push(_y);

          row1.push([item.baseXY![0], item.baseXY![1]]);
          row2.push([current_xy[0], current_xy[1]]);
          row3.push(x, y);
          row4.push(_x, _y);
        }

        const solve = numeric.solve(equations, answers);
        const matrix3d = `matrix3d(
          ${solve[0]}, ${solve[3]}, 0, ${solve[6]},
          ${solve[1]}, ${solve[4]}, 0, ${solve[7]},
                0,           0, 1,               0,
          ${solve[2]},  ${solve[5]}, 0,          1
        )`;

        _this.getWrapperElement().style.transform = matrix3d;
        _this.getWrapperElement().style.zIndex = "10000";

        console.log(matrix3d);

        PIManager.viewer.setUpdate(true);
      }

      // Set Relative Event
      PIManager.viewer.on("animatefinished", this.relativeSizeEvent);
      PIManager.viewer.on("zoomchange", this.relativeSizeEvent);

      this.relativeInit = true;
    }

  }

[aproni34f]

Is there a updated repo for this example? https://github.com/mpetroff/pannellum/issues/974

I would like to 4 dots to be draggable and image to transform as drag happens.

[ChanderShekharKestone]

uld like to 4 d

Till now this is only https://codesandbox.io/s/pannellum-react-57vogl

[aproni34f]

Anyone know how was this made? https://360rumors.com/polygon-hotspots-virtual-tours/

There is a div and a canvas above pnlm-render-container, but I dont see any transforms being applied to these elements, so how does this interactive shape above the piano follow the pannellum movement?

[luishcq]

Isn't this a question better suited to teliportme developers? It seems

pannellum, but it is clear to me that this is another piece of software.

Em sáb., 30 de abr. de 2022 às 19:29, aproni34f @.***> escreveu:

Anyone know how was this made? https://blog.teliportme.com/polygon-hotspots-virtualtour/

There is a div and a canvas above pnlm-render-container, but I dont see any transforms being applied to these elements, so how does this interactive shape in front of the closet follow the pannellum movement?

— Reply to this email directly, view it on GitHub https://github.com/mpetroff/pannellum/issues/974#issuecomment-1114064286, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACPYPLL6YYOMXOX42ZQHGOTVHWX4NANCNFSM4ZU37GFQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- -- Luis Henrique Camargo Quiroz http://luishcq.br.tripod.com - http://www.christusrex.org/www2/cantgreg http://panoramaslh.net/

[aproni34f]

Possibly, but I cannot find github repository for their project. Also, teliportme seems not free like pannellum, so I doubt they share their code.

[mpetroff]

TeliportMe is built using Pannellum. The polygon hot spots are a proprietary extension, which renders the hot spots to a separate transparent <canvas> that is over the Pannellum WebGL <canvas>. I have no knowledge of how it was written.

[dbwodlf3]

I would like to help but i am now working for another project...