Multiple shaders, tile-sources, dynamic behaviour and transparency

[Aiosa]

Hi, thanks for this awesome project!

I can't get tile-drawing to work as transparent. If I register tile-loaded then everything works just fine. But with the default mode, the alpha channel of fragment shader output color is for some reason ignored. And tile-loaded mode introduces some loading-based flickering (the fragments are drawn at incorrect positions and then quickly redrawn at the correct one), which is not applicable in my case. Also, I am using multiple tile sources so the reason I need transparency is to draw some metadata atop an image.

EDIT: this is probably due to OpenSeadragon behaviour because tile-drawing already 'prepared' canvas...

[Aiosa]

To partially handle the transparency issue, if you have multiple sources to work with, use tile-loaded event as openSeadragon did not harm your canvas yet. In openSeadragonGl.js do the following:

this.interface = {
        'tile-loaded': function(e) {
            // process the tile data using webGL
            var output = this.viaGL.toCanvas(e.image, e);
            // DON'T do this, both performance and the result cries
            // e.image.src = output.toDataURL();

            // just add overlay and draw the result into it
            var canvas = document.createElement( 'canvas' )
            canvas.width = e.image.width;
            canvas.height = e.image.height;
            var renderedContext = canvas.getContext('2d');
            renderedContext.drawImage(output, 0, 0);
            // register this so that seadragon knows it's been already rendered
            e.tile.context2D = renderedContext;
        },

You will also want to set the WebGL canvas dimensions so that border tiles are rendered correctly, something like

this.viaGL.setDimensions(e.tile.sourceBounds.width, e.tile.sourceBounds.height);

where setDimensions will change webGL canvas dimensions and update viewport.

EDIT: You probably want to use tile-loaded instead of tile-drawing anyway. The first event is fired only once, so if you don't parse the content into a string and rather set the canvas directly, the behavior is much smoother. It probably does not matter much if you have simple projects but once you start doing expensive things with pixels, it can really show...

[Aiosa]

I somehow managed to get working all the stuff. Maybe in future I will release modified version of this plugin (or share pull request) that can

• use multiple shaders and freely switch between them (even based on the origin of tile that is being drawn, e.g. different shaders for different tile sources)
• use multiple sourceTiles and select which ones are being modified via webGL and which ones are passed directly to OSD (if you don't modify the tile, don't bother re-drawing the data to the same data)
• support transparency
• be able to redraw tiles with different shader / different uniform variables from cache if possible: without re-requesting the tiles from the server (I actually shared this one, see https://github.com/openseadragon/openseadragon/issues/866#issuecomment-820449459)

I will leave this issue open for others to see it easily, if you need similar features just write me a mail and I will share my modifications.

BTW: thanks for this awesome plugin. There are indeed many limitations and it has many problems once you try to do some advanced stuff, but it helped me to get started very quickly!

[iangilman]

Wonderful! Thank you for offering to share your findings with the community :)

[Aiosa]

Ok I had some time, so I tried to put together the updates I made and uploaded them to my own fork of this repository (there are already multiple pull requests and the repo seems to be without maintenance so far....)

Noticed kind of late that there is already a pull request for WebGL 2.0, would incorporate that one as well, maybe later. There might still be some problems (like navigator is not being updated for now, but I think I will be fixing that soon...)

[iangilman]

@Aiosa awesome!

@thejohnhoffer any interest in incorporating those changes into this plugin?

[briandyn]

@Aiosa I'm using your fork of this plugin and for some reason when triggering the redraw() function the tiles become blurry, any Ideas as to why this may be happening?

Example: (BEFORE)

Example: (AFTER)

This seems to be applied with the cacheHandler method

[Aiosa]

Hi unfortunately no. I don't even remember much about the plugin anymore, I was refactoring it more and more and then it was no longer the original plugin. Now I use my own WebGL preprocessor for openseadragon which is part of the WSI viewer I develop (dev branch recommended, will get a merge soon). This renderer is also planned to be incorporated into OpenSeadragon more or less, improving the current WebGL drawer. There is just too little time. I hope in the summer I will be able to move on with the OSD implementation.