DigitalGlobe / gbdxtools

(Deprecated) Python SDK for using GBDX
MIT License
74 stars 57 forks source link

RDA Template Images can't decode some image tiles from RDA #727

Closed drwelby closed 5 years ago

drwelby commented 5 years ago
import os
os.environ["VIRTUAL_RDA_URL"] = "https://rda-dev.geobigdata.io/v1"
from gbdxtools import RDATemplateImage
img = RDATemplateImage('04f10eb3b38fab7c4a81923b527580e9b779815772481174465f9a8cdc05e6fa', 'Orthorectified_Pauli', calibration='sigma0')
win = img.randwindow((500,500))
win.compute()

Error is occuring here:

https://github.com/DigitalGlobe/gbdxtools/blob/master/gbdxtools/rda/fetch/threaded/libcurl/easy.py#L55

arr = imread(temp.name)

imread can't figure out the pixel mode, and is throwing the error 'unknown pixel mode'

drwelby commented 5 years ago

Looks like imread can't decode RDA tiles when:

CatalogImage tiles are fetched with a .tif extension, RDATemplate images have no extension.

If the tile has a .tif extension then imread will use the tifffile plugin, which can read 32-bit tiffs. If there is no extension, then it uses imageio which then fails.

Some possible solutions:

The problem is hinted at here:

https://github.com/DigitalGlobe/gbdxtools/blob/master/gbdxtools/rda/fetch/threaded/libcurl/easy.py#L46

with a TODO that didn't

drwelby commented 5 years ago

Modifying the tile reading to:

try:
    arr = imread(temp.name)
except SyntaxError:
    arr = imread(temp.name, plugin='tifffile')

seems to work. The question is can template nodes return tiles of other image types, and whether imageio can decode them.

drwelby commented 5 years ago

Need to look at sniffing the mimetype of the image

drwelby commented 5 years ago

RDA returns mimetypes for all tiles, so we'll use that to pick the appropriate image loading plugin:

https://github.com/DigitalGlobe/gbdxtools/tree/ISS-727-template-tiles

drwelby commented 5 years ago

Manually merged to 0.16.7

drwelby commented 4 years ago

This error also returns the odd error about Error code: 200; in 0.16.5 the error message is misleading.

There is a workaround for 0.16.5 and earlier versions:

from skimage import io
io.use_plugin('tifffile', 'imread')

This forces the fetcher to use the tifffile plugin to decode the tiff tiles.