Calculate points in original image / projection

martinlombana commented 5 years ago

Hi, I wonder if there is any way to calculate, upon click on the panorama, the exact or aproximate pixel of the original image, taking into account the projection.

If you can just lay out the basics or point me in the right direction, I'll code the rest. But @mpetroff, you will have, for sure, much more experience.

My aim is to be able to delete parts of the underlaying scene image selectively. To paint them black, for instance.

Thanks!

mpetroff commented 5 years ago

The API has a mouseEventToCoords function, which will convert a click event to pitch / yaw. Then it's just a matter of converting to equirectangular image coordinates:

var viewer = pannellum.viewer(...);
viewer.on('mousedown', function(event) {
    // coords[0] is pitch, coords[1] is yaw
    var coords = viewer.mouseEventToCoords(event);
    // convert to image coordinates
    var x = (coords[1] / 360 + 0.5) * imageWidthInPixels;
    var y = (0.5 - coords[0] / 180) * imageHeightInPixels;
    console.log(x, y);
});

The image coordinate lookup gets slightly more complicated if you don't have a full panorama.

martinlombana commented 5 years ago

That's great, I will try it. Thanks a lot! In theory it is only for full 360 panoramas, so this could theoretically work. Thanks for your insight!.

Best!

martinlombana commented 5 years ago

And is there a way to change the pixels of the current base image of the current scene?

I just saw, in the API that the renderer accepts just a configuration object, but no changes after that. I am looking for that, and also to represent 2 layers at the same time that move at the same time, like a sandwidch. This way I will be able to develop a pixelating feature to pixelate portions of a panorama.

Thanks again for the help.

mpetroff commented 5 years ago

You can't in general modify an image from JavaScript anyway, so modifying the original image isn't the correct approach.

I would create an off-screen <canvas> element the size of the image, paint the image into the <canvas>, and then pass the <canvas> as the panorama to Pannellum, setting dynamic: true in the configuration. You can then modify the pixels in the <canvas>, and they will update in the Pannellum viewer, as long as you call viewer.setUpdate(true) before updating the <canvas> (call viewer.setUpdate(false) after you're done to be more efficient). The Video.js plugin works in this manner, except with a <video> element instead of a <canvas> element.

martinlombana commented 5 years ago

That is exactly what I was looking for!

Thanks so much!

martinlombana commented 5 years ago

With this method, we can not really add several scenes with the addScene option, right? I tried adding several scenes with different canvas, but then I get an unknown error.

We shoud add manually the hot spots and handle the scene change by our-selves, if we use dynamic=true?

Thanks,

mpetroff commented 5 years ago

I think using several scenes, each with its own canvas, should work, but I've never tested that. There should be a more detailed error message in the browser's developer console. What does it say?

martinlombana commented 4 years ago

I missed this reply. Sorry.

What I am trying to do now, as I thought it would be better than handling myself the loading and creation of every canvas is the following:

I have several scenes. Each of them point to a different JPG. When I change the scene, everything works as expected.

Now. When I set the viewer to be dynamic: true, no matter what I do, I don't get any errors, but a black screen, when changing the scene. And I am calling viewer.setUpdate(true); when changing the scene, but still that doesn't work.

What I wanted to do is to, when the scene changes, get the reference to the canvas:

/* viewer.setUpdate(true); var canvas = viewer.getCanvas(); console.log(canvas-> , canvas); var ctx = canvas.getContext("2d"); ctx.fillRect(0, 0, 350, 300); */

And then draw a simple square on it. That's it.

On the other hand, in order to be able to draw in the image, without the 3D distorsion, I suppose I have to do what you said, and pass the canvas to the viewer?

martinlombana commented 4 years ago

<!DOCTYPE HTML>
<html>
<head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Tour</title>
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/pannellum@2.5.6/build/pannellum.css"/>
    <script type="text/javascript" src="https://cdn.jsdelivr.net/npm/pannellum@2.5.6/build/pannellum.js"></script>
    <style>
    #panorama {
        width: 600px;
        height: 400px;
    }
    </style>
</head>
<body>

<div id="panorama"></div>
<script>
pannellum.viewer('panorama', {   
    "default": {
        "firstScene": "circle",
        "author": "Matthew Petroff",
        "sceneFadeDuration": 1000,
        "dynamic": true
    },

    "scenes": {
        "circle": {
            "title": "Mason Circle",
            "hfov": 110,
            "pitch": -3,
            "yaw": 117,
            "type": "equirectangular",
            "panorama": "1.jpg",
            "hotSpots": [
                {
                    "pitch": -2.1,
                    "yaw": 132.9,
                    "type": "scene",
                    "text": "Spring House or Dairy",
                    "sceneId": "house"
                }
            ]
        },

        "house": {
            "title": "Spring House or Dairy",
            "hfov": 110,
            "yaw": 5,
            "type": "equirectangular",
            "panorama": "2.jpg",
            "hotSpots": [
                {
                    "pitch": -0.6,
                    "yaw": 37.1,
                    "type": "scene",
                    "text": "Mason Circle",
                    "sceneId": "circle",
                    "targetYaw": -23,
                    "targetPitch": 2
                }
            ]
        }
    }
});

pannellum.viewer.setUpdate(true);
pannellum.viewer.setUpdate(false);

</script>

</body>
</html>

Given this code, it does not load the images anymore.

martinlombana commented 4 years ago

I also took a look at how the video plugin does it. And I am unable to pass a canvas as a panorama. I tried to pass it as blob, but with no success.

I tried both methods. In the current situation, the viewer justs goes to black.

NOTE: I am using the latest Github libs. Attatched testing project, served in a local server.

This is an updated code of 2 canvases being drawn correctly and being passed to the viewer. There are no errors in the console. Only the black screen and a bit of frustration on my part.

Because this is exactly what I need (minus the drawing of the actual panorama images, of course). But If I can get this to work, and pass canvases that I can draw and then update on the viewer, it would be just perfect.

<!DOCTYPE HTML>
<html>

<head>
    <meta charset="utf-8">
    <meta content="width=device-width, initial-scale=1.0" name="viewport">
    <title>Tour</title>
    <link href="pannellum.css" rel="stylesheet" />
    <script src="pannellum.js" type="text/javascript"></script>
    <script src="libpannellum.js" type="text/javascript"></script>
    <style>
        #panorama {
            width: 600px;
            height: 400px;
        }
    </style>
</head>

<body>

    <div id="panorama"></div>

    <script>

        var canvas1 = document.createElement('canvas');
        canvas1.id = "canvas1";
        canvas1.width = 1224;
        canvas1.height = 768;
        canvas1.style.zIndex = 8;
        canvas1.style.position = "absolute";
        canvas1.style.border = "1px solid";
        var body = document.getElementsByTagName("body")[0];
        body.appendChild(canvas1);

        var canvas2 = document.createElement('canvas');
        canvas2.id = "canvas2";
        canvas2.width = 1224;
        canvas2.height = 768;
        canvas2.style.zIndex = 8;
        canvas2.style.position = "absolute";
        canvas2.style.border = "1px solid";
        canvas2.style.left = "444px";

        var body = document.getElementsByTagName("body")[0];
        body.appendChild(canvas2);

        c = document.getElementById("canvas1");
        var ctx = c.getContext("2d");
        ctx.fillStyle = "rgba(255, 0, 0, 0.2)";
        ctx.fillRect(100, 100, 200, 200);
        ctx.fillStyle = "rgba(0, 255, 0, 0.2)";
        ctx.fillRect(150, 150, 200, 200);
        ctx.fillStyle = "rgba(0, 0, 255, 0.2)";
        ctx.fillRect(200, 50, 200, 200);

        var config = {};
        config.scenes = {}
        config.type = 'equirectangular';
        config.dynamic = true;

        var viewer = pannellum.viewer('panorama', config);

        viewer.addScene('s1', {
            'type': 'equirectangular',
            'panorama': canvas1
        });
        viewer.addScene('s2', {
            'type': 'equirectangular',
            'panorama': canvas2
        });

        viewer.loadScene('s1');

        viewer.setUpdate(true);

        c = document.getElementById("canvas1");
        var ctx = c.getContext("2d");
        ctx.fillStyle = "rgba(5, 30, 240, 0.2)";
        ctx.fillRect(300, 100, 600, 200);
        ctx.fillStyle = "rgba(0, 0, 0, 0.2)";
        ctx.fillRect(450, 150, 200, 200); 

        viewer.setUpdate(false);

    </script>

</body>

</html>

martinlombana commented 4 years ago

I finally managed it... with timers.

Loading a scene inmediatly after adding the canvases, does not work. This way, it works:

<!DOCTYPE HTML>
<html>

<head>
    <meta charset="utf-8">
    <meta content="width=device-width, initial-scale=1.0" name="viewport">
    <title>Tour</title>
    <link href="pannellum.css" rel="stylesheet" />
    <script src="pannellum.js" type="text/javascript"></script>
    <script src="libpannellum.js" type="text/javascript"></script>
    <style>
        #panorama {
            width: 600px;
            height: 400px;
        }

        canvas{
            display: block;
        }
    </style>
</head>

<body>

    <div id="panorama"></div>

    <script>

        var canvas1 = document.createElement('canvas');
        canvas1.id = "canvas1";
        canvas1.width = 1224;
        canvas1.height = 768;
        canvas1.style.zIndex = 8;
        canvas1.style.position = "absolute";
        canvas1.style.border = "1px solid";
        var body = document.getElementsByTagName("body")[0];
        body.appendChild(canvas1);

        var canvas2 = document.createElement('canvas');
        canvas2.id = "canvas2";
        canvas2.width = 1224;
        canvas2.height = 768;
        canvas2.style.zIndex = 8;
        canvas2.style.position = "absolute";
        canvas2.style.border = "1px solid";
        canvas2.style.left = "444px";

        var body = document.getElementsByTagName("body")[0];
        body.appendChild(canvas2);

        c = document.getElementById("canvas1");
        var ctx = c.getContext("2d");
        ctx.fillStyle = "rgba(255, 0, 0, 0.2)";
        ctx.fillRect(100, 100, 200, 200);
        ctx.fillStyle = "rgba(0, 255, 0, 0.2)";
        ctx.fillRect(150, 150, 200, 200);
        ctx.fillStyle = "rgba(0, 0, 255, 0.2)";
        ctx.fillRect(200, 50, 200, 200);

        c = document.getElementById("canvas1");
        var ctx = c.getContext("2d");
        ctx.fillStyle = "rgba(5, 0, 0, 0.6)";
        ctx.fillRect(300, 100, 600, 200);

        var config = {};
        config.scenes = {}
        config.type = 'equirectangular';
        config.dynamic = true;
        config.sceneFadeDuration = 800;
        config.strings = {
            'loadButtonLabel': 'Haz click para <br>cargar<br> el panorama.',
            'loadingLabel': 'Cargando...',
            'bylineLabel': 'by %s',
            'noPanoramaError': 'Panorama no encontrado.',
            'fileAccessError': 'El archivo %s no puede ser accedido.',
            'malformedURLError': 'La url del panorama no está bien formada.',
            'iOS8WebGLError': 'Debido a OS8 y su implementaci´pn de WebGL, sólo se pueden usar JPGs codificados de forma progresiva.',
            'genericWebGLError': 'Tu navegador no tiene las características necesarias para ver este panorama.',
            'textureSizeError': 'Este panorama es demasiado grande! Tiene %spx de ancho, pero tu dispositivo sólo soporta %spx.',
            'unknownError': 'Error desconocido.'
        }

        var viewer = pannellum.viewer('panorama', config);

        viewer.addScene('s1', {
            'type': 'equirectangular',
            'panorama': canvas1
        });
        viewer.addScene('s2', {
            'type': 'equirectangular',
            'panorama': canvas2
        });

        setTimeout(() => {
            viewer.loadScene('s1');
            viewer.setUpdate(true);
            viewer.setUpdate(false);

        }, 1000)

        setTimeout(() => {

            viewer.setUpdate(true);

            c = document.getElementById("canvas1");
            var ctx = c.getContext("2d");
            ctx.fillStyle = "rgba(5, 30, 240, 0.2)";
            ctx.fillRect(300, 100, 600, 200);
            ctx.fillStyle = "rgba(0, 0, 0, 0.2)";
            ctx.fillRect(450, 150, 200, 200);

            viewer.setUpdate(false);

        }, 6000)

    </script>

</body>

</html>

There is no callback for when the scenes have been successfully added, or a promise-like or observable-like system that could be added in order to avoid this pitfall. @mpetroff ?

Thanks!

mpetroff commented 4 years ago

The addScene method is synchronous. All it does is add an entry to the internal configuration object. There is no need for a promise.

Regardless, I don't see any reason for either the <canvas> element to be created dynamically or for the scenes to be added dynamically. It would be simpler and less error prone if the <canvas> elements were part of the HTML and the scenes were part of the initial configuration.

fahome10bd commented 2 years ago

To explain my situation, my panorama camera is on my car, and it produces yaw, pitch and roll according to the camera orientation. The produced panorama is shown using Pannellum viewer. But from the pannellum viewer the yaw and pitch I get from click using the above formula I tried to convert it to track the pixel position. But unfortunately, the pitch value is giving error almost 20-60 pixels. The yaw is close to the actual value. The error is larger when the car pitch and roll is higher. Any idea to solve this issue so that I can get the pixel position value of the clicked point from pannellum viewer?

mpetroff / pannellum

Calculate points in original image / projection #765