Load event visibly fires before multi resolution panoramas are loaded

[DStillingfleet]

Hi Matthew

I'm exploring the use of multi resolution panoramas.

I've noticed that the Load event fires before even the lowest resolution tiles have loaded, i.e. you can still see the black background for a moment, just as you can see it when the example loads at https://pannellum.org/documentation/examples/multiresolution/.

I'm using the Load event to reveal the panorama once it's loaded so as to produce a smoother scene transition, and to ensure other functions do not run until the panorama is ready.

Any suggestions as to how to overcome this?

Best wishes

Derek

[mpetroff]

This is the expected behavior. It was either very briefly show a black screen or very briefly show a loading screen, and I decided that the black screen was less jarring. If you use a fade between scenes, this usually isn't noticeable, except on very slow connections. As part of a new multires format (discussed in https://github.com/mpetroff/pannellum/issues/911#issuecomment-688561874), I plan on including an extremely low resolution placeholder image in the JSON configuration, to show instead of just a black screen.

[fluidadsjtr]

I see #911 is closed but we are keen to have support for the low res-image to avoid the back flicker.

I notice that you say "If you use a fade between scenes, this usually isn't noticeable, except on very slow connections", but I have a high speed connection and have used fade, yet the flicker is still very visible. We tried a slower fade and it does not help.

Can we change code to made fade wait for image load, or maybe we can sponsor the code to allow a low resolution plceholder image in the JSON configuration (that would work well I think).

[mpetroff]

I see #911 is closed but we are keen to have support for the low res-image to avoid the back flicker.

911 was not related to the issue you're describing; it was related to performance while panning / zooming. I mentioned some future plans in a comment on that issue, which is why it was linked to from here.

Can we change code to made fade wait for image load

It's free / open source source software, so there's nothing stopping you from doing so. However, I don't think this is the best idea, since it will appear as if the viewer has frozen, given the lack of a loading screen. If you wanted to do this anyway, you could start by adding this method to libpannellum.js (it's something that I had previously written for a bespoke integration):

    this.isBaseLoaded = function() {
        if (program.currentNodes.length >= 6) {
            for (var i = 0; i < 6; i++) {
                if (!program.currentNodes[i].textureLoaded) {
                    return false;
                }
            }
            return true;
        }
        return false;
    }

It will let you know once the base tiles have finished loading, at which point there's no longer any black background. Then, in pannellum.js, you could move the if (config.sceneFadeDuration && renderer.fadeImg !== undefined) {...} block from the renderInitCallback function to the animate function and change the if statement to if (config.sceneFadeDuration && renderer.fadeImg !== undefined && renderer.isBaseLoaded()) {...}. This should wait until the base tiles have finished loading before starting the fade, but I haven't tested it.

or maybe we can sponsor the code to allow a low resolution plceholder image in the JSON configuration

I've thought more about the best method for implementing a low-resolution placeholder since I previously mentioned plans for including one, and I think the best method will be to modify BlurHash to use spherical harmonic transforms instead of discrete cosine transforms, which would allow it to be used for panoramas. The resulting string, containing a few dozen characters, could then be included in the configuration file and used as a zero-load-time preview until the base multires tiles are loaded, replacing the black screen. However, I'm a bit short on time at the moment, so I don't know when I'll be able to work on it.

[fluidadsjtr]

Hi my developer has been struggling with this, so if you are interested in this work getting sponsored let me know,

[fluidadsjtr]

Hi, My developer has not been able to fix this, I thought I would follow up and let you know I am still keen to sponsor the development if you now have time (or the near future)?

I liked you suggested idea, but an alternative is that we do a zoom in the current scene before changing scene this would seem to provide an opportunity to load the next image(s) whilst the zoom takes place?

[DStillingfleet]

Hi John, Matthew

I've been given this some thought to the early firing of the on load event, and whilst I've not had the opportunity to try them yet, maybe there is an alternative to on load.

scenechange Fired when a scene change is initiated. A load event will be fired when the new scene finishes loading. Passes scene ID string to handler.

scenechangefadedone If a scene transition fade interval is specified, this event is fired when the fading is completed after changing scenes.

I'm not sure how different scenechange is from load, but it might be worth testing.

scenechangefadedone could perhaps be used by setting the fade duration to a small number and seeing the difference between it and the on load event.

What do you think Matthew?

[fluidadsjtr]

Hi,

In multires load event fires as soon as scenechange fired because multires does not wait to load image

Sent from my iPhone

On 6 Jun 2021, at 17:03, Derek Stillingfleet @.***> wrote:

 Hi John, Matthew

I've been given this some thought to the early firing of the on load event, and whilst I've not had the opportunity to try them yet, maybe there is an alternative to on load.

scenechange Fired when a scene change is initiated. A load event will be fired when the new scene finishes loading. Passes scene ID string to handler.

scenechangefadedone If a scene transition fade interval is specified, this event is fired when the fading is completed after changing scenes.

I'm not sure how different scenechange is from load, but it might be worth testing.

scenechangefadedone could perhaps be used by setting the fade duration to a small number and seeing the difference between it and the on load event.

What do you think Matthew?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[mpetroff]

@fluidadsjtr I should have time in July, but the next three weeks are very busy for me.

You should be able to implement your suggested zoom method using a custom hot spot click handler function. It could trigger a zoom using the setHfov method and then use XHR requests to initiate downloads of the six base tiles of the new scene. It could then use the loadScene method to load the new scene after a delay equal to the animation time used when calling setHfov. At this point the browser should have the base tiles cached, so it shouldn't need to re-download them when the scene is actually loaded. The alternative would be to modify the scene fade code to first zoom into the view of the old scene before fading it out, but this is less than ideal due to the quality issues involved with zooming into an image beyond 100% zoom.

@DStillingfleet As @fluidadsjtr pointed out, the issue is that the viewer considers the load of a multires scene to be instant because there is no loading screen. I've never consider this to be a significant issue, but not everyone holds this opinion.

[fluidadsjtr]

Hi Matthew,

We have tried both methods, however, we are rotating the screen as we open and as such as it rotates the "black" flickers in. What would be ideal would be to preload the canvas with a low resolution equirectangular image, That would then mean the image is there and not blackness, which would then get detail added. This would remove the flicker feeling.

John

[fluidadsjtr]

Hi Matthew,

Is there an way to make it load a lower res file that then is updated with a high res file ? I am making the multi res files so I can easy change the file sizes getting created.

[fluidadsjtr]

@mpetroff "I should have time in July, but the next three weeks are very busy for me.", let me know when your available.

Thanks

[mpetroff]

Is there an way to make it load a lower res file that then is updated with a high res file?

With regard to multires panoramas, this isn't possible, but you can decrease the multires tile size, which will decrease the size of the base tiles. The default is 512px, but 256px is also a reasonable size.

With regard to equirectangular images, you can do this if you handle image loading yourself, draw the images to a <canvas>, pass the <canvas> as the panorama image to Pannellum, and set dynamic: true; use the setUpdate() method when the canvas changes.

let me know when you're available

I should have some time next week.

I also found some time to make a proof of concept for encoding panoramas with spherical harmonics. Here's what the result looks like when the image used from the simple example is encoded and then decoded: https://bl.ocks.org/mpetroff/raw/0f5bfb93e537429da7119538a00748a6/

The data would be stored as a 74 character string in the configuration file (unlike the demo, which stores the rendered preview as an image) and would be rendered from the first frame, eliminating the black screen.

[fluidadsjtr]

Hopefully you will get time to do Tile (or even better allow a low res rectangular image to be provided for multires). Let me know the costs and how to pay.

Thanks

John

On Fri, 7 May 2021 at 01:40, Matthew Petroff @.***> wrote:

I see #911 https://github.com/mpetroff/pannellum/issues/911 is closed but we are keen to have support for the low res-image to avoid the back flicker.

911 https://github.com/mpetroff/pannellum/issues/911 was not related

to the issue you're describing; it was related to performance while panning / zooming. I mentioned some future plans in a comment on that issue, which is why it was linked to from here.

Can we change code to made fade wait for image load

It's free / open source source software, so there's nothing stopping you from doing so. However, I don't think this is the best idea, since it will appear as if the viewer has frozen, given the lack of a loading screen. If you wanted to do this anyway, you could start by adding this method to libpannellum.js (it's something that I had previously written for a bespoke integration):

this.isBaseLoaded = function() {
    if (program.currentNodes.length >= 6) {
        for (var i = 0; i < 6; i++) {
            if (!program.currentNodes[i].textureLoaded) {
                return false;
            }
        }
        return true;
    }
    return false;
}

It will let you know once the base tiles have finished loading, at which point there's no longer any black background. Then, in pannellum.js, you could move the if (config.sceneFadeDuration && renderer.fadeImg !== undefined) {...} block from the renderInitCallback function to the animate function and change the if statement to if (config.sceneFadeDuration && renderer.fadeImg !== undefined && renderer.isBaseLoaded()) {...}. This should wait until the base tiles have finished loading before starting the fade, but I haven't tested it.

or maybe we can sponsor the code to allow a low resolution plceholder image in the JSON configuration

I've thought more about the best method for implementing a low-resolution placeholder since I previously mentioned plans for including one, and I think the best method will be to modify BlurHash https://blurha.sh/ to use spherical harmonic transforms instead of discrete cosine transforms, which would allow it to be used for panoramas. The resulting string, containing a few dozen characters, could then be included in the configuration file and used as a zero-load-time preview until the base multires tiles are loaded, replacing the black screen. However, I'm a bit short on time at the moment, so I don't know when I'll be able to work on it.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/mpetroff/pannellum/issues/912#issuecomment-833974460, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGLYUCPNXX2VEK5ZWTXJKSDTMMZHJANCNFSM4RDHTRYA .

[mpetroff]

Support for multires SHT and thumbnail previews was added in 9788fc6512fe0de950cb10678dc4d4f453ad376e.

The multires generate.py script now automatically generates spherical-harmonic-transform-based previews for spherical panoramas, assuming the pyshtools Python package is installed. It will also generate a thumbnail-based preview if --thumbnailsize N is specified as an argument, where N is a positive integer. If the higher-fidelity thumbnail-based previews are desired, I'd recommend trying N = 256 and seeing if that is enough resolution; there's no hard upper limit for the resolution, but I wouldn't recommend using N > 512, since it increases the configuration file size without providing significant gains.