search

insarlab / MintPy

Miami InSAR time-series software in Python
https://mintpy.readthedocs.io
Other
575 stars 252 forks source link

Mintpy can't load `int8` types from GDAL #1081

Closed scottstanie closed 11 months ago

scottstanie commented 1 year ago

Description of the problem

int8 data types were added in GDAL 3.7. These are being used by isce3 as the layover/shadow mask datatype. They're not recognizable by mintpy due to the hard coded mappings between GDAL and numpy datatypes.

Full script that generated the error

$ gdalinfo geometry/layover_shadow_mask.tif  | tail -3
Center      (  768960.000, 3564540.000) (102d 8'50.20"W, 32d11' 6.97"N)
Band 1 Block=168x48 Type=Int8, ColorInterp=Gray
  NoData Value=0

In [1]: from mintpy.utils import readfile
In [2]: readfile.read_attribute('geometry/layover_shadow_mask.tif')

Full error message

Traceback (most recent call last):
  File "/u/aurora-r0/staniewi/miniconda3/envs/mapping-311/lib/python3.11/site-packages/ipdb/__main__.py", line 318, in main
    pdb._run(stdlib_pdb._ScriptTarget(mainpyfile))
  File "/u/aurora-r0/staniewi/miniconda3/envs/mapping-311/lib/python3.11/pdb.py", line 1643, in _run
    self.run(target.code)
  File "/u/aurora-r0/staniewi/miniconda3/envs/mapping-311/lib/python3.11/bdb.py", line 600, in run
    exec(cmd, globals, locals)
  File "<string>", line 1, in <module>
  File "/u/aurora-r0/staniewi/repos/sweets/scripts/prep_mintpy.py", line 586, in <module>
    main(sys.argv[1:])
  File "/u/aurora-r0/staniewi/repos/sweets/scripts/prep_mintpy.py", line 570, in main
    prepare_geometry(geom_file, geom_dir=inps.geom_dir, metadata=meta)
  File "/u/aurora-r0/staniewi/repos/sweets/scripts/prep_mintpy.py", line 393, in prepare_geometry
    dsDict[dsName] = readfile.read(fname, datasetName=dsName)[0]
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/u/aurora-r0/staniewi/repos/MintPy/src/mintpy/utils/readfile.py", line 290, in read
    atr = read_attribute(fname, datasetName=dsname4atr)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/u/aurora-r0/staniewi/repos/MintPy/src/mintpy/utils/readfile.py", line 1248, in read_attribute
    atr.update(read_gdal_vrt(metafile))
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/u/aurora-r0/staniewi/repos/MintPy/src/mintpy/utils/readfile.py", line 1630, in read_gdal_vrt
    atr['DATA_TYPE'] = DATA_TYPE_GDAL2NUMPY[ds.GetRasterBand(1).DataType]
                       ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 14
Uncaught exception. Entering post mortem debugging

The one line fix

add 14: 'int8', to the DATA_TYPE_GDAL2NUMPY and 'int8': 14 to DATA_TYPE_NUMPY2GDAL

another possible fix:

Alternative to the quick fix- take a function like these instead of the dict mappings: https://github.com/opera-adt/dolphin/blob/main/src/dolphin/utils.py#L69-L102 so that they use the GDAL functions

yunjunz commented 1 year ago

The gdal int8 was hardwired already, but not correctly. I like your approach in dolphin. Could you issue a PR for it in mintpy also?

We could still update the mapping dict, just for bookkeeping purposes.

ditafaith commented 7 months ago

Hi @yunjunz

I have installed mintpy and everything looks normal. However, when running it gets the following error. I'm not sure if it's related to GDAL, since I have it installed. Look forward to getting your hint.

raise ImportError('Cannot import gdal and osr!')
ImportError: Cannot import gdal and osr!
yunjunz commented 7 months ago

Hi @yunjunz

I have installed mintpy and everything looks normal. However, when running it gets the following error. I'm not sure if it's related to GDAL, since I have it installed. Look forward to getting your hint.

raise ImportError('Cannot import gdal and osr!')
ImportError: Cannot import gdal and osr!

The error shows that your GDAL can not be imported.

https://github.com/insarlab/MintPy/blob/1d25397fc50b1027d8280db77752d6ba60f8411b/src/mintpy/utils/readfile.py#L1661

ditafaith commented 7 months ago

Is there any suggestion what should I do ? Meanwhile I check with the basic command such as gdalinfo it's work.

scottstanie commented 7 months ago

Perhaps open a different issue about it? Although it is a very likely not a Mintpy issue, but rather a python setup problem with how you have installed GDAL

ditafaith commented 7 months ago

As I mentioned, I can ran gdal command without problem. However, may be my question not directly related to this threads and can be closed.

yunjunz commented 7 months ago

GDAL and GDAL's Python binding are two different things. Please ensure your gdal and mintpy are installed in the same Python environment, and gdal can be imported in that environment. This had been an issue a long time ago, but I have not seen it for many years now.

ditafaith commented 7 months ago

Thanks @yunjunz for the response.