Openseadragonfillter CONVOLUTION grid line issue

[PpyMohan]

After Adding CONVOLUTION Openseadragonfillter i am getting grid lines on image how to hide this grid lines ?? And it is clearly visible when i zoom image

[iangilman]

Yes, that's a known issue with this plugin. See the "Edge Effects" note at the bottom here: https://github.com/usnistgov/OpenSeadragonFiltering#edge-effects Also see #4.

Of course it would be great to fix! The challenge is that this filter operates on each tile individually, and the tiles don't have access to the pixels of their neighbors' edges. A couple possible strategies come to mind:

• Figure out how to correlate the tiles with each other so the edges of other tiles can be included in the calculation. This could either be done ad hoc, so the calculation happens across canvases, or we could have another offscreen canvas where we lay the tiles out together for the purpose of running the convolution and then copy the results back into the individual tiles.
• Don't run the convolution on the tiles at all, but run it instead on the viewer's canvas when it updates.

Of these approaches, I imagine the offscreen canvas would be the best performance, but it would also require the most memory.

One trade-off to be aware of is if you run the convolution on the tiles, its scale changes as you zoom (and again as each tile layer is revealed), whereas if you run it on the viewer's canvas, the convolution scale remains that of the canvas regardless of how you zoom. Which result is more appealing depends on the application.

[PpyMohan]

Is that possible to remove that grid line ??

[iangilman]

It would take a bunch of development work; it's not easy. My suggestions for how to do that work are in the comment above.

[rmd13]

It would take a bunch of development work; it's not easy. My suggestions for how to do that work are in the comment above.

Hi, Is the current filter run on the viewer's canvas, or on the source tiles? I encountered problems that if I load too many filters and drag the canvas too fast, then some black square appeared and cannot be filled no matter how I change the filter, zoom or drag the canvas. From this bug and the source code of context.getImageData, it seems that the the filter was run on the viewer's canvas. Thus if a blask square was not filled before the execution of getImageData, that black square will be treated as pixels and undergo following image filtering and make it black forever. Am I right?

If so, what might be the source of black square? I though it woule be the syn/asyn problems. To solve this problem, I change all the demo.js and demo-bundle.js source code to make all the statement of asyn converted to syn, then I found that the filtering was slower and less responsive, but the black square problem improved a lot.

However, there is still ocasional black square, although much less than before. I though this problem is also dependent on on the CPU speed and Graphic card. Better CPU and GPU can solve it. Am I right?

Thanks.

[iangilman]

I believe the filter is run on each tile's canvas (each tile has its own canvas that the original tile image is copied to for performance reasons), not the viewer's canvas.

It's hard to say where these black squares are coming from. Someone might be able to take a look at it if you post a demo of it in action (on CodePen or some such).

[rmd13]

The black square problem is not easy to reproduce because the code is under nomal code. It is just that when I zoom and drag to fast, then black square sometimes appear. I use the same js code as the filter demo website. The only difference is that the code is not run on a server, but run locally by open html files in IE and enable the blocked danger. Chrome cannot display the image on the html content.

[iangilman]

Have you tried running your code on a local server rather than just opening the file? That should help with Chrome displaying the image, I believe.

[18073479643]

May I ask whether the grid line problem has been solved? This problem is very important to me

[iangilman]

Not that I'm aware of. You're welcome to work on a fix if you'd like.

What's your scenario? Perhaps there are workarounds…

[18073479643]

I used caman.js sharpening and the resulting picture had grid lines

[18073479643]

I think it was caused by the sharpening of the tiles, which sharpened the edges of the tiles. After the tiles were joined together, there were very obvious gaps

[18073479643]

If I don't describe my problem clearly, can I send you an email to further describe the problem

[iangilman]

Makes sense; that's a classic case of the issue discussed here. The true fix would be the code improvements mentioned near the top of this thread.

If you wanted to try to hack around it, one possibility would be to not use this plugin, but instead run a sharpen filter directly on the OpenSeadragon canvas after every time it draws. I have no idea if that would look good or perform well, but it's something to try.

Of course, another option would be to sharpen your image on the server before converting it into tiles.

[18073479643]

I have tried the method you mentioned, but it doesn't work and will end in failure. After dragging the OpenSeadragon canvas, the sharpening effect will be lost, which I think is the result of creating a new canvas while dragging. If you loop through the sharpening effect, this will cause the page to get stuck, and there are many problems with using this method that I won't list

[iangilman]

Okay, it was worth a try! Is it possible for you to sharpen your image before converting it to tiles?

Also, how big is your image? Maybe it doesn't need to be tiled?

[18073479643]

Here's how it works. My images are small images stored in folders, and I use OpenSeadragon to access the folder path and splice them together into a larger image. These steps are required and cannot be changed. I used Openseadragonfillter and CamanJS to sharpen it, and the grid lines came out. I think the sharpening is the operation before the small pictures are stitched, and the stitching is done after the small pictures are sharpened, which causes the grid line problem. I have also tried many methods (including the direct sharpening of canvas instead of small pictures as you said), but none of them have solved my problem well, so I would like to ask for your help

[18073479643]

I can't sharpen the image before turning it into a block. Do you mean the size of the tile or the size of the Mosaic canvas? The width of the canvas is uncertain, based on the width of the computer monitor (around 1100) and the height is 840px. The tiles are 256px wide and 256px high

[iangilman]

If the image you're sending to OSD is only 1100 x 840, that's definitely small enough you don't actually need to tile it. I'd recommend using https://openseadragon.github.io/examples/tilesource-legacy/ or https://openseadragon.github.io/examples/tilesource-image/. Then you should be able to sharpen it all at once using the filter plugin as before, but without the seams.

[18073479643]

I don't quite understand what you mean. I followed the two links you gave me and found that there seems to be nothing I need to modify. If I just change the value of buildPyramid to false, I think I have done it, but my picture doesn't seem to cause any change

[iangilman]

Can you share what your tile source code looks like when you put it in the viewer?

[18073479643]

viewer = OpenSeadragon({ id: "sliderOpenseadragon", prefixUrl: "xxx/openseadragon/images/", constrainDuringPan: true, showNavigator: true, navigatorPosition: "TOP_RIGHT" tileSources: { Image: { xmlns: "xxx",
Url: xxx + "/",
buildPyramid:false,
Overlap: "0",
TileSize: "256",
Format: "jpg",

            Size: {
                Height: xxx,
                Width: xxx
            },
            tileOverlap: 0
        },
    },
    showNavigationControl: false,
    springStiffness: 1, 
    animationTime: 1, 
    maxImageCacheCount: 1000 ,
});

[iangilman]

Thank you. Looks like you're using a DZI. What I'm suggesting is instead of a DZI, create just a single JPG, and then use the format specified in https://openseadragon.github.io/examples/tilesource-image/ to load it.

[18073479643]

I'm trying, but I haven't succeeded yet. I want to ask a question, according to your method this method, https://openseadragon.github.io/examples/tilesource-image/, will only a picture display? If so, I don't think so. Because my picture is more than 300 small pictures, spliced into a big picture, and finally displayed on the canvas.

[iangilman]

How big is the big picture? When I asked you before, you said 1100 x 840, but maybe you meant that's the size for the small pictures?

[18073479643]

Yes, the big picture is 1100 x 840. But my picture is made up of a number of small pictures pieced together, the method of https://openseadragon.github.io/examples/tilesource-image/, you don't seem support splicing display pictures really this way, support only display a full photo. Of course, I also have the full large image, i.e. no need to splice. I also try to https://openseadragon.github.io/examples/tilesource-image/, although after sharpening showed no grid lines, but a new problem. When I zoomed in on the image, there were a lot of pixels and the overall image was significantly less sharp, which was a poor user experience. Obviously, this is not possible.

[iangilman]

Was that with buildPyramid: true? It might be worth trying it both ways.

Another thing to try would be using https://openseadragon.github.io/examples/tilesource-legacy/ with a few image sizes. Basically, a pyramid but not broken into tiles at each level.

For that matter, another approach would be to stick with DZI, but make your tile size 1100. That way, you get the pyramid but no level is broken into tiles.

I'm suggesting all of these workarounds on the assumption that you're not up for going in and fixing the issue in this plugin (as described near the top of this thread), as it would probably be pretty involved. Also, with your big picture being such a reasonable size, you don't need the tiling anyway. Of course if you are interested in fixing it, go for it!

[18073479643]

Ok, thank you very much, I will try this method https://openseadragon.github.io/examples/tilesource-legacy/，but you may have some misunderstand what I mean. Splicing small tiles into a larger picture is what I needed, although it caused the grid lines to appear after the image was sharpened. Now I just want to get rid of the gridlines, but I don't want to change the way I used to splice the smaller images to show the larger ones, because it's very important and necessary for me

[iangilman]

I understand. Once it's spliced into a larger image, OSD doesn't care that it was originally a bunch of smaller images. It's just going to treat it as a single image. Are you able to do the splicing on the server side?

[18073479643]

It's hard to say, I have to talk to my colleagues about it, because the server side is not my responsibility. But I can tell you that there is a reason why we chose to slice pictures. If a complete picture is returned directly at the server side, it will be very slow when displayed at the client side, and it will be stuck when dragging the canvas, which greatly affects the experience of the customer. Therefore, we chose to slice the picture and then splice it at the front end. Now, if we go back to the server and splice the images and transfer them, I think we might have a similar problem again, which we don't want

[18073479643]

I would like to ask you another question. When I use DZI, I will add an image name after this line of code Url: xxx + "/", that is, an image name will be spelled after this "/". Is this added by OpenSeadragon for me? If yes, https://openseadragon.github.io/examples/tilesource-legacy/ this method can also like DZI behind the URL to add a picture name for me?

[iangilman]

Your complete pictures shouldn't be slow if they're only 1100 pixels wide. Is it possible for you to share one, or is it company secret?

The legacy tile source doesn't know anything about DZI; you would have to use exact image URLs.

[18073479643]

It's just a picture. It's not a company secret. I can assure you, however, that it did cause delays in client loading and loss of image sharpness because we started with the whole image, and because of these problems, we had to cut up the image on the server side and splicing it together on the WEB side. Of course, it may just be that on our system, images load slowly. But anyway, this is already an unchangeable thing, because the version has been iterated for a long time, we will have to use DZI, if change now, may cause some other problems, and whether to change the method, do not use DZI, It's not up to me

[18073479643]

I feel a little hopeless, and at this point, it may not be possible to solve this problem. If so, we'll have to forgo sharpening. But anyway,I really appreciate your patient explanation. Thank you very much

[iangilman]

Sure... Sorry I couldn't solve the problem for you! If you or your company are willing to dig in and put some engineering into the fixes I suggest at the top of the thread (or pay someone else to), it must be solvable, but I expect it'll take a good deal of work.

[18073479643]

I think I may have found a solution, but there are still some problems. I wonder if you have a solution

[18073479643]

I have written a set of image sharpening algorithm, this can achieve better sharpening effect. I sharpened the canvas generated with DZI, but when the canvas was dragged, the sharpening effect disappeared. I think this may be because the canvas became a new canvas after being dragged, so my effect disappeared. Does DZI have an API to prevent the canvas from being automatically updated when dragged

[18073479643]

In other words, whether my sharpening algorithm can be put into viewer.setFilterOptions, and whether viewer.setFilterOptions supports its own methods. I've tried but failed, and I don't know if I'm trying the wrong way or if viewer.setFilterOptions itself isn't supported

[iangilman]

Are you sharpening the canvas directly? That seems like a good solution. You just have to do it every frame, because when the user drags, the canvas is redrawn. Have your code watch for the viewer's update-viewport event and sharpen each time.

[18073479643]

This won't do. Sharpening algorithms are expensive, and using update-viewport causes drag delays. I was wondering, is there any way I can sharpen the DZI tiles instead of sharpening the entire canvas, like Openseadragonfillter

[iangilman]

Sharpening the DZI tiles is what gets the grid lines we're trying to get rid of here. That said, you could sharpen the image before you turn it into tiles. But I assume you want to be able to sharpen it interactively?

One potential solution is the WebGL drawer that @pearcetm is working on with https://github.com/openseadragon/openseadragon/pull/2310. It's going to be a while before that is ready to release, but you might be interested to see it in progress. You should be able to do a real-time sharpening with WebGL.