Image sizing/positioning issues

rsimon commented 6 years ago

Hi,

I'm experimenting with IIIF images from lots of different sources, and I'm experiencing different display issues (unfortunately, on almost all of them). Images don't appear properly centered in the viewer, and they are not properly fit to the screen bounds. It's as if the size/resolution is not properly picked up from the info.json.

I collected a few examples:

https://recogito.pelagios.org/document/z8gcpxgb9ycawu/part/1/edit - this image doesn't show initially. Scroll out a few times to see it.
https://recogito.pelagios.org/document/b6jzpya906fhzv/part/1/edit - same here, no image intially, only after scrolling out a few times (after which the image is in ultra-high resolution)
https://recogito.pelagios.org/document/pnud7opghdw2rv/part/1/edit - this image does show initally, but is neither properly centered nor fit.
For comparison: https://recogito.pelagios.org/document/cyka3e1xrsi0ih/part/1/edit - the sample from the Klokan demo page works just fine

For each example, I linked the info.json into the page, so you can take a look at what it contains. (It's the link in the header.)

I know the IIIF viewer might not currently be at the top of your priority list. But it would be great if you could take a look. If there's anything I can do to provide further information, or help out otherwise - do let me know!

Disclaimer: I'm running my own fork of the code, but have verified that the parts that relate to width/height/tilesize/resolution etc. are still identical to the code in this repo.

rsimon commented 6 years ago

P.S.: I have another instructive example, where the image is offset exactly by half it's width and height. I.e. the initial center of the viewing are is the bottom-right corner of the image; and if you click zoom, the image will zoom around that bottom right corner (=center of the viewport). I can't share this example publicly (copyright), but we can get in touch via E-Mail.

rsimon commented 6 years ago

P.P.S.: I suspect the manifest info might sometimes even be wrong occasionally? I wonder how Mirador and UV manage to deal with them, though. Could it be that they are often just bypassing the info.json files and rely on the item manifests only?

petrsloup commented 6 years ago

On the first look, the issue is probably with different tile sizes (different than 256x256). Some of the images use much larger tiles (1024x1024) and some even non-square tiles (2713x1910, which is currently not supported at all).

The viewer tries to respect the tile sizes (in case there are pre-created images on the server), but there is probably some issue in the calculation of resolutions and zoom levels.

rsimon commented 6 years ago

It looks like the tilesize is properly picked up from the manifest. (E.g. I checked for the 1024 pixel-tile image). And it's also forwared properly into the ol.tilegrid.TileGrid as the tileSize argument.

var tileGrid: new ol.tilegrid.TileGrid({
        resolutions: logicalResolutions.reverse(),
        origin: [ 0, 0 ],
        tileSize: logicalTileSize // Checked that this is 1024 for the test image
      })

From what I can see, everything else is handled by OpenLayers itself, so I'm puzzled that the offset still persists. The function that renders the image to the canvas tile (the one that gets set via setTileLoadFunction()) seems to work correctly, too. Could this be a bug in OpenLayers?

rsimon commented 6 years ago

Not sure if this helps at all, but I have a case where tilesize is default 256x256. The manifest doesn't specify anything at all:

{  
   "profile":[  
      "http://iiif.io/api/image/2/level2.json",
      {  
         "supports":[  
            "canonicalLinkHeader",
            "profileLinkHeader",
            "mirroring",
            "rotationArbitrary",
            "sizeAboveFull"
         ],
         "qualities":[  
            "default",
            "color",
            "gray",
            "bitonal"
         ],
         "formats":[  
            "jpg",
            "png",
            "gif",
            "webp"
         ]
      }
   ],
   "protocol":"http://iiif.io/api/image",
   "sizes":[],
   "height":5785,
   "width":4596,
   "@context":"http://iiif.io/api/image/2/context.json",
   "@id":"https://www.example.com/0d4a6d1e-20dc-4ddd-80ea-2a035dc9dcad-000001"
}

Yet the offset happens in the view:

screenshot

When clicking the zoom button, the image zooms around the bottom-right corner; and I've fitted the view to full extent of the image (i.e. actually, the image should be zoomed to twice the size, centered at the middle of the viewport).

I'm initializing the projection like so:

new ol.proj.Projection({
            code: 'IIIF',
            units: 'pixels',
            extent: [0, -h, w, 0]
          }

and the view:

          view: new ol.View({
            projection: projection,
            zoom: 0,
            minResolution: 0.125
          })

which, as far as I can see, is identical to how it's done in your code.

rsimon commented 6 years ago

Hi,

I'm still trying to get to the bottom of this issue. It looks as if it's related to the calculation of the maxZoom parameter and/or how it is being used to compute the tile URL arguments. I'm really just doing trial and error (i.e. poking at things with a stick and seeing if anything happens). But I did get to display most of my examples correctly by setting

var maxZoom = Math.floor(Math.log(tileSize, 2));

Doesn't seem right though, since the maxZoom now has nothing to do with the image width and height. Yet the way this var is used as a factor in computing the tile URL does seem to generate the right results.

The only two examples that are still broken for me are

one that does have non-square tiles (actually a single image if I saw correctly), i.e. that's not supposed to work given current limitations
another one that only declares tile width (of 1024px) in the manifest. Therefore the tile height is automatically set to 256 and, hence, it's another case of non-square tiles (and, in fact, a broken manifest, since the tile seem actually 1024x1024).

If you have any thoughts/hints about the role of the maxZoom arg in the tile URL - do let me know!

UPDATE: P.S.: here's the URL again for the square-tiles sample that was broken initially, but now works: https://recogito.pelagios.org/document/pnud7opghdw2rv/part/1/edit

rsimon commented 6 years ago

Hm, unfortunately: another example that doesn't work with the above "fix":

https://recogito.pelagios.org/document/6bu41oopbkejlg

In this case, tiles are 256x256. And, worse: in this case there's also a mismatch between the image coordinates computed by OpenLayers, and the actual image dimension :-(

rsimon commented 6 years ago

Had a chance to look into this in some more detail now. My previous hack regarding maxZoom was bollocks of course, but the problem does seem related to maxZoom after all. I compared code with the OpenSeadragon IIIF source implementation, which handles my examples fine. The small crucial difference seems to be how they compute that parameter:

https://github.com/openseadragon/openseadragon/blob/master/src/iiiftilesource.js#L131

I.e. they are rounding normally, not upward with Math.ceil(). I changed that in you code, and that seemed to do the trick!

Some of my examples are still broken. But as far as I can see, these are all related to non-square tiles. Adding support for this should be pretty manageable in your implementation as well. I might give it a shot at some point if a have the time.

UPDATE: Hm, sorry, still no luck. The fix does improve the sitation in some cases, but not all. I have an image with size 5785x4596 -> displays fine. And an image 5856x4600 -> rendered one zoom level too small, as in above screenshot.

It's worth noting that it's not just a display issue; OpenLayers (according to the coordinates) does consider the whole viewport area "the image", for some reason it just picks the wrong level for fetching the tiles. (I can actually confirm that in the tileUrlFunction: the tileUrlFunction receives a z tile coord of 0, when the correct current zoom level would actually be 1.)

rsimon commented 6 years ago

Hi again,

ok, traced this a bit more. One thing that didn't seem to work reliably in your code is to derive the scale from the current zoomlevel and maxZoom:

https://github.com/klokantech/iiifviewer/blob/master/src/iiifsource.js#L93

I replaced this by getting the scale from the resolutions array instead & that seems to work reliably.

rsimon commented 6 years ago

Hi, just quickly checking back. This now seems to work properly for the most part in our fork. Primarily following the change to the computation of the scale parameter in the tile URL:

var scale = logicalResolutions[z] / devicePixelRatio;

Full source file is here: https://github.com/pelagios/recogito2/blob/master/app/assets/javascripts/document/annotation/image/iiif/iiifSource.js

lutzhelm commented 5 years ago

The problem occurs with images where the scaleFactors in the IIIF image info.json contain more values than necessary to fit the image into the tilesize. In that case, scaleFactors does not only contain values from 2^0 to 2^maxZoom but also additional values that scale the image even smaller.

@rsimon: If I am not mistaken, your solution fixes the problem that the scale that OpenLayers expects and the scale that the source provides do not fit together. But there remains the issue that the source will not offer tiles for all values in resolutions.

klokantech / iiifviewer

Image sizing/positioning issues #23