search

ruven / iipsrv

iipsrv is an advanced high-performance feature-rich image server for web-based streamed viewing and zooming of ultra high-resolution images.
https://iipimage.sourceforge.io
GNU General Public License v3.0
293 stars 115 forks source link

Can only read first tile, otherwise "TPTImage :: TIFFReadEncodedTile() failed" #206

Open arildm opened 3 years ago

arildm commented 3 years ago

With any request that includes more than the first tile, I get an error:

Request URI mapped to iiif=test/medelbild/vips2/9118326.tif/0,0,257,256/full/0/default.jpg
Full Request is iiif=test/medelbild/vips2/9118326.tif/0,0,257,256/full/0/default.jpg
[1/1]: Command / Argument is iiif : test/medelbild/vips2/9118326.tif/0,0,257,256/full/0/default.jpg
IIIF handler reached
IIIF :: URL decoded to test/medelbild/vips2/9118326.tif/0,0,257,256/full/0/default.jpg
FIF handler reached
FIF :: URL decoding/filtering: test/medelbild/vips2/9118326.tif => test/medelbild/vips2/9118326.tif
FIF :: Image cache hit. Number of elements: 1
FIF :: TIFF image detected
FIF :: Created image
FIF :: Image dimensions are 3779 x 5711
FIF :: Image contains 3 channels with 8 bits per channel
FIF :: Image timestamp: Fri, 05 Mar 2021 12:45:28 GMT
FIF :: Total command time 705 microseconds
IIIF :: Requested Region: x:0, y:0, w:257, h:256
IIIF :: Requested Size: 257x256
IIIF :: Requested Rotation: 0 degrees
IIIF :: Requested Quality: default with format: jpg
IIIF :: image request for test/medelbild/vips2/9118326.tif with arguments: region: 0,0,257,256; size: 257x256; rotation: 0; mirroring: 0
CVT handler reached
CVT :: Using resolution 5 with size 3779x5711
CVT :: Region: 0,0,257,256
CVT :: Requested scaled region size is 257x256. Nearest existing resolution is 5 which has region with size 257x256
TileManager getRegion :: Total tiles in image: 15x23 tiles
TileManager getRegion :: Tile start: 0,0 with offset: 0,0
TileManager getRegion :: Tile end: 1,0
TileManager :: Cache Miss for resolution: 5, tile: 0
TileManager :: Cache Size: 2 tiles, 0.197143 MB
TileManager :: Tile cache insertion time: 474 microseconds
TileManager :: Total Tile Access Time: 1912 microseconds
TileManager getRegion :: Tile access time 1959 microseconds for tile 0 at resolution 5
TileManager getRegion :: Tile data is 3 channels, 8 bits per channel
TileManager getRegion :: destination tile width: 256, tile height: 256
TileManager :: Cache Miss for resolution: 5, tile: 1
TileManager :: Cache Size: 3 tiles, 0.389468 MB
TPTImage :: TIFFReadEncodedTile() failed for /data/cdhdata/public/test/medelbild/vips2/9118326.tif
Sending HTTP 404 Not Found
Total Request Time: 4939 microseconds
image closed and deleted
Server count is 7

As you can see, I am requesting region 0,0,257,256 above. With 0,0,256,256, all is fine.

The tiff image is converted from jpeg, I have tried a few different ways with the same result:

I can open the resulting images in GIMP, where I can see the entire image.

I have run the conversion locally (mac OS, libtiff 4.2.0) and running IIP on a server (CentOS, libtiff 4.0.3). I'm working on doing conversion and testing on the same system, but cannot do it right now.

Other than my guess that the libtiff versions are different, does anyone have any idea on what could cause the read error?

ruven commented 3 years ago

There may be something in your Mac OS libtiff causing a problem. Can you paste the output from tiffinfo (you can run it either on your Mac or the CentOS box)?

tiffinfo <path to>/9118326.tif

Is this happening for this specific image or for others too? Which version of iipsrv are you using? And can you copy and paste the section of the iipsrv log file where you say the request works for the first tile from this image?

arildm commented 3 years ago

Here's log entries for another file:

Region: 0,0,256,256 (fine) ``` Request URI mapped to iiif=test/medelbild/vips/9118331.tif/0,0,256,256/full/0/default.jpg Full Request is iiif=test/medelbild/vips/9118331.tif/0,0,256,256/full/0/default.jpg [1/1]: Command / Argument is iiif : test/medelbild/vips/9118331.tif/0,0,256,256/full/0/default.jpg IIIF handler reached IIIF :: URL decoded to test/medelbild/vips/9118331.tif/0,0,256,256/full/0/default.jpg FIF handler reached FIF :: URL decoding/filtering: test/medelbild/vips/9118331.tif => test/medelbild/vips/9118331.tif FIF :: Image cache hit. Number of elements: 1 FIF :: TIFF image detected FIF :: Created image FIF :: Image dimensions are 3761 x 5659 FIF :: Image contains 3 channels with 8 bits per channel FIF :: Image timestamp: Fri, 05 Mar 2021 12:41:47 GMT FIF :: Total command time 2273 microseconds IIIF :: Requested Region: x:0, y:0, w:256, h:256 IIIF :: Requested Size: 256x256 IIIF :: Requested Rotation: 0 degrees IIIF :: Requested Quality: default with format: jpg IIIF :: image request for test/medelbild/vips/9118331.tif with arguments: region: 0,0,256,256; size: 256x256; rotation: 0; mirroring: 0 JTL handler reached JTL :: Setting physical resolution of tile to 1181.1 x 1181.1 pixels/cm JTL :: Embedding ICC profile with size 560 bytes TileManager :: Cache Miss for resolution: 5, tile: 0 TileManager :: Cache Size: 1 tiles, 0.192324 MB TileManager :: JPEG Compression Time: 2579 microseconds TileManager :: Tile cache insertion time: 28 microseconds TileManager :: Total Tile Access Time: 4257 microseconds JTL :: Tile size: 256 x 256 JTL :: Channels per sample: 3 JTL :: Bits per channel: 8 JTL :: Data size is 6787 bytes JTL :: Total command time 4377 microseconds IIIF :: Total command time 6831 microseconds Total Request Time: 7039 microseconds image closed and deleted Server count is 4 ```
Region: 257,0,10,10 (same error) ``` Request URI mapped to iiif=test/medelbild/vips/9118331.tif/257,0,10,10/full/0/default.jpg Full Request is iiif=test/medelbild/vips/9118331.tif/257,0,10,10/full/0/default.jpg [1/1]: Command / Argument is iiif : test/medelbild/vips/9118331.tif/257,0,10,10/full/0/default.jpg IIIF handler reached IIIF :: URL decoded to test/medelbild/vips/9118331.tif/257,0,10,10/full/0/default.jpg FIF handler reached FIF :: URL decoding/filtering: test/medelbild/vips/9118331.tif => test/medelbild/vips/9118331.tif FIF :: Image cache hit. Number of elements: 1 FIF :: TIFF image detected FIF :: Created image FIF :: Image dimensions are 3761 x 5659 FIF :: Image contains 3 channels with 8 bits per channel FIF :: Image timestamp: Fri, 05 Mar 2021 12:41:47 GMT FIF :: Total command time 3058 microseconds IIIF :: Requested Region: x:257, y:0, w:10, h:10 IIIF :: Requested Size: 10x10 IIIF :: Requested Rotation: 0 degrees IIIF :: Requested Quality: default with format: jpg IIIF :: image request for test/medelbild/vips/9118331.tif with arguments: region: 257,0,10,10; size: 10x10; rotation: 0; mirroring: 0 CVT handler reached CVT :: Using resolution 5 with size 3761x5659 CVT :: Region: 257,0,10,10 CVT :: Requested scaled region size is 10x10. Nearest existing resolution is 5 which has region with size 10x10 TileManager getRegion :: Total tiles in image: 15x23 tiles TileManager getRegion :: Tile start: 1,0 with offset: 1,0 TileManager getRegion :: Tile end: 1,0 TileManager :: Cache Miss for resolution: 5, tile: 1 TileManager :: Cache Size: 2 tiles, 0.199277 MB TPTImage :: TIFFReadEncodedTile() failed for /data/cdhdata/public/test/medelbild/vips/9118331.tif Sending HTTP 404 Not Found Total Request Time: 4130 microseconds image closed and deleted Server count is 5 ```

And the startup output:

IIPImage Server. Version 1.1
*** Ruven Pillay <ruven@users.sourceforge.net> ***

Verbosity level set to 10
Running in FCGI mode

Setting maximum image cache size to 100MB
Setting filesystem prefix to '/data/cdhdata/public/'
Setting filesystem suffix to ''
Setting default JPEG quality to 75
Setting maximum CVT size to 3000
Setting HTTP Cache-Control header to 'max-age=86400'
Setting 3D file sequence name pattern to '_pyr_'
Setting IIIF version to 2
Setting Cross Origin Resource Sharing to '*'
Setting base URL to [redacted]
Setting Allow Upscaling to true
Setting ICC profile embedding to true
Setting up JPEG2000 support via OpenJPEG
Setting image processing engine to CPU processor
OpenMP enabled for parallelized image processing with 4 threads
Setting URI mapping to iiif=>IIIF. Supported protocol: IIIF
tiffinfo vips/9118326.tif ``` TIFF Directory at offset 0x2fbbea (3128298) Image Width: 3779 Image Length: 5711 Tile Width: 256 Tile Length: 256 Resolution: 1181.1, 1181.1 pixels/cm Bits/Sample: 8 Sample Format: unsigned integer Compression Scheme: JPEG Photometric Interpretation: YCbCr Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Reference Black/White: 0: 0 255 1: 128 255 2: 128 255 ICC Profile: , 560 bytes JPEG Tables: (142 bytes) TIFF Directory at offset 0x3f8ba0 (4164512) Subfile Type: reduced-resolution image (1 = 0x1) Image Width: 1889 Image Length: 2855 Tile Width: 256 Tile Length: 256 Resolution: 1181.1, 1181.1 pixels/cm Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: YCbCr Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Reference Black/White: 0: 0 255 1: 128 255 2: 128 255 ICC Profile: , 560 bytes JPEG Tables: (142 bytes) TIFF Directory at offset 0x4498ea (4495594) Subfile Type: reduced-resolution image (1 = 0x1) Image Width: 944 Image Length: 1427 Tile Width: 256 Tile Length: 256 Resolution: 1181.1, 1181.1 pixels/cm Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: YCbCr Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Reference Black/White: 0: 0 255 1: 128 255 2: 128 255 ICC Profile: , 560 bytes JPEG Tables: (142 bytes) TIFF Directory at offset 0x462296 (4596374) Subfile Type: reduced-resolution image (1 = 0x1) Image Width: 472 Image Length: 713 Tile Width: 256 Tile Length: 256 Resolution: 1181.1, 1181.1 pixels/cm Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: YCbCr Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Reference Black/White: 0: 0 255 1: 128 255 2: 128 255 ICC Profile: , 560 bytes JPEG Tables: (142 bytes) TIFF Directory at offset 0x469d56 (4627798) Subfile Type: reduced-resolution image (1 = 0x1) Image Width: 236 Image Length: 356 Tile Width: 256 Tile Length: 256 Resolution: 1181.1, 1181.1 pixels/cm Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: YCbCr Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Reference Black/White: 0: 0 255 1: 128 255 2: 128 255 ICC Profile: , 560 bytes JPEG Tables: (142 bytes) TIFF Directory at offset 0x46c8c2 (4638914) Subfile Type: reduced-resolution image (1 = 0x1) Image Width: 118 Image Length: 178 Tile Width: 256 Tile Length: 256 Resolution: 1181.1, 1181.1 pixels/cm Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: YCbCr Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Reference Black/White: 0: 0 255 1: 128 255 2: 128 255 ICC Profile: , 560 bytes JPEG Tables: (142 bytes) ```
tiffinfo im/9118326.tif ``` TIFF Directory at offset 0x803f12 (8404754) Subfile Type: multi-page document (2 = 0x2) Image Width: 3779 Image Length: 5711 Tile Width: 256 Tile Length: 256 Resolution: 300, 300 pixels/inch Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: RGB color FillOrder: msb-to-lsb Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Page Number: 0-7 White Point: 0.3127-0.329 PrimaryChromaticities: 0.640000,0.330000,0.300000,0.600000,0.150000,0.060000 ICC Profile: , 560 bytes JPEG Tables: (73 bytes) TIFF Directory at offset 0xb13846 (11614278) Subfile Type: multi-page document (2 = 0x2) Image Width: 1889 Image Length: 2855 Tile Width: 256 Tile Length: 256 Resolution: 150, 150 pixels/inch Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: RGB color FillOrder: msb-to-lsb Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Page Number: 0-7 White Point: 0.3127-0.329 PrimaryChromaticities: 0.640000,0.330000,0.300000,0.600000,0.150000,0.060000 ICC Profile: , 560 bytes JPEG Tables: (73 bytes) TIFF Directory at offset 0xc14b38 (12667704) Subfile Type: multi-page document (2 = 0x2) Image Width: 944 Image Length: 1427 Tile Width: 256 Tile Length: 256 Resolution: 75, 75 pixels/inch Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: RGB color FillOrder: msb-to-lsb Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Page Number: 0-7 White Point: 0.3127-0.329 PrimaryChromaticities: 0.640000,0.330000,0.300000,0.600000,0.150000,0.060000 ICC Profile: , 560 bytes JPEG Tables: (73 bytes) TIFF Directory at offset 0xc62e7e (12988030) Subfile Type: multi-page document (2 = 0x2) Image Width: 472 Image Length: 713 Tile Width: 256 Tile Length: 256 Resolution: 37.5, 37.5 pixels/inch Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: RGB color FillOrder: msb-to-lsb Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Page Number: 0-7 White Point: 0.3127-0.329 PrimaryChromaticities: 0.640000,0.330000,0.300000,0.600000,0.150000,0.060000 ICC Profile: , 560 bytes JPEG Tables: (73 bytes) TIFF Directory at offset 0xc80814 (13109268) Subfile Type: multi-page document (2 = 0x2) Image Width: 236 Image Length: 356 Tile Width: 256 Tile Length: 256 Resolution: 18.75, 18.75 pixels/inch Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: RGB color FillOrder: msb-to-lsb Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Page Number: 0-7 White Point: 0.3127-0.329 PrimaryChromaticities: 0.640000,0.330000,0.300000,0.600000,0.150000,0.060000 ICC Profile: , 560 bytes JPEG Tables: (73 bytes) TIFF Directory at offset 0xc91146 (13177158) Subfile Type: multi-page document (2 = 0x2) Image Width: 118 Image Length: 178 Tile Width: 256 Tile Length: 256 Resolution: 9.375, 9.375 pixels/inch Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: RGB color FillOrder: msb-to-lsb Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Page Number: 0-7 White Point: 0.3127-0.329 PrimaryChromaticities: 0.640000,0.330000,0.300000,0.600000,0.150000,0.060000 ICC Profile: , 560 bytes JPEG Tables: (73 bytes) TIFF Directory at offset 0xca2298 (13247128) Subfile Type: multi-page document (2 = 0x2) Image Width: 59 Image Length: 89 Tile Width: 256 Tile Length: 256 Resolution: 4.6875, 4.6875 pixels/inch Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: RGB color FillOrder: msb-to-lsb Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Page Number: 0-7 White Point: 0.3127-0.329 PrimaryChromaticities: 0.640000,0.330000,0.300000,0.600000,0.150000,0.060000 ICC Profile: , 560 bytes JPEG Tables: (73 bytes) ```
arildm commented 3 years ago

I finally managed to install vips on the server, and when I use that to convert images, IIP reads the resulting files successfully.

It would be interesting to know if you could help me find what the problem is with the Mac-generated tiffs, but it's not prioritized for me anymore. Feel free to close the issue if you want to.

The tiffinfo output for the server-generated tiff: (Resolution and JPEG Tables lines are different)

tiffinfo (server) ``` TIFF Directory at offset 0x30694a (3172682) Image Width: 3779 Image Length: 5711 Tile Width: 256 Tile Length: 256 Resolution: 300, 300 pixels/inch Bits/Sample: 8 Sample Format: unsigned integer Compression Scheme: JPEG Photometric Interpretation: YCbCr Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Reference Black/White: 0: 0 255 1: 128 255 2: 128 255 ICC Profile: , 560 bytes JPEG Tables: (574 bytes) TIFF Directory at offset 0x405ff8 (4218872) Subfile Type: reduced-resolution image (1 = 0x1) Image Width: 1889 Image Length: 2855 Tile Width: 256 Tile Length: 256 Resolution: 300, 300 pixels/inch Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: YCbCr Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Reference Black/White: 0: 0 255 1: 128 255 2: 128 255 ICC Profile: , 560 bytes JPEG Tables: (574 bytes) TIFF Directory at offset 0x4575c2 (4552130) Subfile Type: reduced-resolution image (1 = 0x1) Image Width: 944 Image Length: 1427 Tile Width: 256 Tile Length: 256 Resolution: 300, 300 pixels/inch Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: YCbCr Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Reference Black/White: 0: 0 255 1: 128 255 2: 128 255 ICC Profile: , 560 bytes JPEG Tables: (574 bytes) TIFF Directory at offset 0x47009a (4653210) Subfile Type: reduced-resolution image (1 = 0x1) Image Width: 472 Image Length: 713 Tile Width: 256 Tile Length: 256 Resolution: 300, 300 pixels/inch Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: YCbCr Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Reference Black/White: 0: 0 255 1: 128 255 2: 128 255 ICC Profile: , 560 bytes JPEG Tables: (574 bytes) TIFF Directory at offset 0x477b7c (4684668) Subfile Type: reduced-resolution image (1 = 0x1) Image Width: 236 Image Length: 356 Tile Width: 256 Tile Length: 256 Resolution: 300, 300 pixels/inch Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: YCbCr Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Reference Black/White: 0: 0 255 1: 128 255 2: 128 255 ICC Profile: , 560 bytes JPEG Tables: (574 bytes) TIFF Directory at offset 0x47a6ce (4695758) Subfile Type: reduced-resolution image (1 = 0x1) Image Width: 118 Image Length: 178 Tile Width: 256 Tile Length: 256 Resolution: 300, 300 pixels/inch Bits/Sample: 8 Compression Scheme: JPEG Photometric Interpretation: YCbCr Orientation: row 0 top, col 0 lhs Samples/Pixel: 3 Planar Configuration: single image plane Reference Black/White: 0: 0 255 1: 128 255 2: 128 255 ICC Profile: , 560 bytes JPEG Tables: (574 bytes) ```
ruven commented 3 years ago

Strange. Could you send me a copy of one of the TIFF images from your Mac that does not work with iipsrv?

Ricky-Y commented 3 years ago

Hi @ruven , I had same issue when playing around with iipsrv and iipmooviewer on my mac.

When I use imagemagick to generate .tiff, I got this error, when I use libvips to generate .tiff, it works, although there is still a slight issue that one tile was not placed perfectly, but at least the full image was shown. Do you want me to copy the .tiff I generated? I can provide that to you.

ruven commented 3 years ago

Hi @Ricky-Y, yes please send me a link to the TIFF to see what's going wrong.

Ricky-Y commented 3 years ago

@ruven I uploaded the original photo and two generated tif files in https://drive.google.com/drive/folders/1TQgfyYlfZJUBkEEwz5_B_V2v9mDpRHHF?usp=sharing

ruven commented 2 years ago

Hi @Ricky-Y, both of your images work fine for me on Linux. There must be a problem with the version of libtiff installed on your Mac, which may not have libtiff configured to use JPEG.