search

ASFHyP3 / hyp3-isce2

HyP3 plugin for ISCE2 processing
Apache License 2.0
10 stars 8 forks source link

Unable to write ISCE format on MacOS (SystemError: Unkown GDAL Error) #178

Closed scottyhq closed 6 months ago

scottyhq commented 6 months ago

The bug

For version 0.9.2. The example workflow fails saving the glo30 DEM in ISCE format running via a conda-install on MacOS. It seems like the conda-forge gdal doesn't have a functional ISCE driver because writing to other formats is fine.

https://github.com/ASFHyP3/hyp3-isce2/blob/5713e162728b9e2b87d8ddc61c2a37de8183b406/src/hyp3_isce2/dem.py#L123-L124

To Reproduce

conda activate hyp3-isce2
python -m hyp3_isce2 ++process insar_tops_burst \
  S1_136231_IW2_20200604T022312_VV_7C85-BURST \
  S1_136231_IW2_20200616T022313_VV_5D11-BURST \
  --looks 20x4 \
  --apply-water-mask False

Additional context

Seems a problem only on MacOS, conda-installs on linux are working fine.

(hyp3-isce2) ➜  conda list                      
gdal                      3.6.3           py311h619941e_1    conda-forge
libgdal                   3.6.3                h8ea55aa_1    conda-forge
rasterio                  1.3.6           py311hc41c901_0    conda-forge
2024-01-12 08:54:12,127 - hyp3_isce2.insar_tops_burst - INFO - DEM ROI: (53.07855985074144, 27.491831927404057, 54.15583739250667, 27.84714385769884)
Reading glo_30 Datasets: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  2.11it/s]
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/scott/GitHub/hyp3-isce2/src/hyp3_isce2/__main__.py", line 51, in <module>
    main()
  File "/Users/scott/GitHub/hyp3-isce2/src/hyp3_isce2/__main__.py", line 47, in main
    sys.exit(process_entry_point.load()())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/scott/GitHub/hyp3-isce2/src/hyp3_isce2/insar_tops_burst.py", line 482, in main
    isce_output_dir = insar_tops_burst(
                      ^^^^^^^^^^^^^^^^^
  File "/Users/scott/GitHub/hyp3-isce2/src/hyp3_isce2/insar_tops_burst.py", line 105, in insar_tops_burst
    dem_path = download_dem_for_isce2(dem_roi, dem_name='glo_30', dem_dir=dem_dir, buffer=0, resample_20m=False)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/scott/GitHub/hyp3-isce2/src/hyp3_isce2/dem.py", line 130, in download_dem_for_isce2
    with rasterio.open(dem_path, 'w', **dem_profile) as ds:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/scott/miniconda3/envs/hyp3-isce2/lib/python3.11/site-packages/rasterio/env.py", line 451, in wrapper
    return f(*args, **kwds)
           ^^^^^^^^^^^^^^^^
  File "/Users/scott/miniconda3/envs/hyp3-isce2/lib/python3.11/site-packages/rasterio/__init__.py", line 314, in open
    dataset = writer(
              ^^^^^^^
  File "rasterio/_io.pyx", line 1450, in rasterio._io.DatasetWriterBase.__init__
  File "rasterio/_err.pyx", line 222, in rasterio._err.exc_wrap_pointer
SystemError: Unkown GDAL Error. To debug: https://rasterio.readthedocs.io/en/latest/topics/errors.html#debugging-internal-gdal-functions
jhkennedy commented 6 months ago

Hmm, @forrestfwilliams, since you use a Mac, can you look at this?

forrestfwilliams commented 6 months ago

Hey @scottyhq are you on an Intel or ARM based Mac? This will change our debugging efforts.

forrestfwilliams commented 6 months ago

@scottyhq I wasn't able to re-create the error you're seeing on my 2021 M1 MacBook Pro.

Just FYI, hyp3-isce2 won't run correctly natively on an ARM platform, because there is no ARM-based version of ISCE2 available on conda-forge.

To get around this, you can set up a pseudo-Intel conda environment for hyp3-isce2 by running the commands below in hyp3-isce2's home directory:

export CENV_NAME=hyp3-isce2
export CONDA_SUBDIR=osx-64
export OSX_SDK_DIR=/Library/Developer/CommandLineTools/SDKs
conda create -y -n $CENV_NAME
conda activate $CENV_NAME
conda config --env --set subdir osx-64
mamba env update --name $CENV_NAME -f environment.yml

Try installing hyp3-isce2 this way and let me know how it goes.

scottyhq commented 6 months ago

Also on a 2019 intel. I just followed the development instructions, so maybe it's a version issue with some new build?... no rush on debugging, just wanted to flag it

conda environment ``` name: hyp3-isce2 channels: - conda-forge dependencies: - affine=2.4.0=pyhd8ed1ab_0 - aom=3.5.0=hf0c8a7f_0 - asf_search=6.7.3=pyhd8ed1ab_0 - attrs=23.2.0=pyh71513ae_0 - blosc=1.21.5=heccf04b_0 - boost-cpp=1.78.0=h31500c2_2 - boto3=1.34.17=pyhd8ed1ab_0 - botocore=1.34.17=pyhd8ed1ab_0 - branca=0.7.0=pyhd8ed1ab_1 - brotli=1.1.0=h0dc2134_1 - brotli-bin=1.1.0=h0dc2134_1 - brotli-python=1.1.0=py311hdf8f085_1 - bzip2=1.0.8=h10d778d_5 - c-ares=1.25.0=h10d778d_0 - ca-certificates=2023.11.17=h8857fd0_0 - cached-property=1.5.2=hd8ed1ab_1 - cached_property=1.5.2=pyha770c72_1 - cairo=1.16.0=h904041c_1014 - certifi=2023.11.17=pyhd8ed1ab_0 - cfitsio=4.2.0=hd56cc12_0 - cftime=1.6.3=py311hc9a392d_0 - charset-normalizer=3.3.2=pyhd8ed1ab_0 - click=8.1.7=unix_pyh707e725_0 - click-plugins=1.1.1=py_0 - cligj=0.7.2=pyhd8ed1ab_1 - cloudpickle=3.0.0=pyhd8ed1ab_0 - colorama=0.4.6=pyhd8ed1ab_0 - contourpy=1.2.0=py311h7bea37d_0 - coverage=7.4.0=py311he705e18_0 - curl=8.1.2=hbee3ae8_0 - cycler=0.12.1=pyhd8ed1ab_0 - cytoolz=0.12.2=py311h2725bcf_1 - dask-core=2023.12.1=pyhd8ed1ab_0 - dateparser=1.2.0=pyhd8ed1ab_0 - dem_stitcher=2.5.4=pyhd8ed1ab_0 - exceptiongroup=1.2.0=pyhd8ed1ab_2 - expat=2.5.0=hf0c8a7f_1 - ffmpeg=5.1.2=gpl_h8b4fe81_106 - fftw=3.3.10=nompi_h4fa670e_108 - fiona=1.9.3=py311h9687163_0 - flake8=7.0.0=pyhd8ed1ab_0 - flake8-blind-except=0.2.1=pyhd8ed1ab_0 - flake8-builtins=2.2.0=pyhd8ed1ab_0 - flake8-import-order=0.18.2=pyhd8ed1ab_0 - folium=0.15.1=pyhd8ed1ab_0 - font-ttf-dejavu-sans-mono=2.37=hab24e00_0 - font-ttf-inconsolata=3.000=h77eed37_0 - font-ttf-source-code-pro=2.038=h77eed37_0 - font-ttf-ubuntu=0.83=h77eed37_1 - fontconfig=2.14.2=h5bb23bf_0 - fonts-conda-ecosystem=1=0 - fonts-conda-forge=1=0 - fonttools=4.47.2=py311he705e18_0 - freetype=2.12.1=h60636b9_2 - freexl=1.0.6=hb7f2c08_1 - fsspec=2023.12.2=pyhca7485f_0 - gdal=3.6.3=py311h619941e_1 - geopandas=0.14.2=pyhd8ed1ab_0 - geopandas-base=0.14.2=pyha770c72_0 - geos=3.11.2=hf0c8a7f_0 - geotiff=1.7.1=h6d03f96_7 - gettext=0.21.1=h8a4c099_0 - giflib=5.2.1=hb7f2c08_3 - gmp=6.3.0=h93d8f39_0 - gnutls=3.7.9=h1951705_0 - graphite2=1.3.13=h2e338ed_1001 - h5py=3.8.0=nompi_py311h777de7a_100 - harfbuzz=6.0.0=h08f8713_0 - hdf4=4.2.15=h9804679_6 - hdf5=1.12.2=nompi_h48135f9_101 - hyp3lib=2.0.2=pyhd8ed1ab_0 - icu=70.1=h96cf925_0 - idna=3.6=pyhd8ed1ab_0 - imagecodecs-lite=2019.12.3=py311h4a70a88_7 - imageio=2.33.1=pyh8c1a49c_0 - importlib-metadata=7.0.1=pyha770c72_0 - importlib_metadata=7.0.1=hd8ed1ab_0 - iniconfig=2.0.0=pyhd8ed1ab_0 - isce2=2.6.3=py311h3432358_0 - jasper=4.0.0=h794afb9_1 - jinja2=3.1.3=pyhd8ed1ab_0 - jmespath=1.0.1=pyhd8ed1ab_0 - joblib=1.3.2=pyhd8ed1ab_0 - json-c=0.16=h01d06f9_0 - kealib=1.5.0=h5c1f988_0 - kiwisolver=1.4.5=py311h5fe6e05_1 - krb5=1.20.1=h049b76e_0 - lame=3.100=hb7f2c08_1003 - lazy_loader=0.3=pyhd8ed1ab_0 - lcms2=2.15=h2dcdeff_1 - lerc=4.0.0=hb486fe8_0 - libaec=1.1.2=he965462_1 - libblas=3.9.0=20_osx64_openblas - libbrotlicommon=1.1.0=h0dc2134_1 - libbrotlidec=1.1.0=h0dc2134_1 - libbrotlienc=1.1.0=h0dc2134_1 - libcblas=3.9.0=20_osx64_openblas - libcurl=8.1.2=hbee3ae8_0 - libcxx=16.0.6=hd57cbcb_0 - libdeflate=1.17=hac1461d_0 - libedit=3.1.20191231=h0678c8f_2 - libev=4.33=h10d778d_2 - libexpat=2.5.0=hf0c8a7f_1 - libffi=3.4.2=h0d85af4_5 - libgdal=3.6.3=h8ea55aa_1 - libgfortran=5.0.0=13_2_0_h97931a8_1 - libgfortran5=13.2.0=h2873a65_1 - libglib=2.78.1=h6d9ecee_0 - libiconv=1.17=hd75f5a5_2 - libidn2=2.3.4=hb7f2c08_0 - libjpeg-turbo=2.1.5.1=h0dc2134_1 - libkml=1.3.0=hab3ca0e_1016 - liblapack=3.9.0=20_osx64_openblas - liblapacke=3.9.0=20_osx64_openblas - libnetcdf=4.9.1=nompi_hb17d2b5_102 - libnghttp2=1.58.0=h64cf6d3_0 - libopenblas=0.3.25=openmp_hfef2a42_0 - libopencv=4.7.0=py311hbc2f7b1_2 - libopus=1.3.1=hc929b4f_1 - libpng=1.6.39=ha978bb4_0 - libpq=15.2=h3640bf0_0 - libprotobuf=3.21.12=h7d26f99_2 - librttopo=1.1.0=h5c328d2_13 - libspatialindex=1.9.3=he49afe7_4 - libspatialite=5.0.1=h595bb97_24 - libsqlite=3.44.2=h92b6c6a_0 - libssh2=1.11.0=hd019ec5_0 - libtasn1=4.19.0=hb7f2c08_0 - libtiff=4.5.0=hd920806_5 - libunistring=0.9.10=h0d85af4_0 - libvpx=1.11.0=he49afe7_3 - libwebp-base=1.3.2=h0dc2134_0 - libxcb=1.15=hb7f2c08_0 - libxml2=2.10.3=h201ad9d_4 - libxslt=1.1.37=h5d22bc9_0 - libzip=1.10.1=hc158999_3 - libzlib=1.2.13=h8a1eda9_5 - llvm-openmp=17.0.6=hb6ac08f_0 - locket=1.0.0=pyhd8ed1ab_0 - lxml=4.9.2=py311h9f2bb26_0 - lz4-c=1.9.4=hf0c8a7f_0 - mapclassify=2.6.1=pyhd8ed1ab_0 - markupsafe=2.1.3=py311h2725bcf_1 - matplotlib-base=3.8.2=py311hd316c10_0 - mccabe=0.7.0=pyhd8ed1ab_0 - munch=4.0.0=pyhd8ed1ab_0 - munkres=1.1.4=pyh9f0ad1d_0 - ncurses=6.4=h93d8f39_2 - netcdf4=1.6.3=nompi_py311he6f34b2_100 - nettle=3.9.1=h8e11ae5_0 - networkx=3.2.1=pyhd8ed1ab_0 - nspr=4.35=hea0b92c_0 - nss=3.96=ha05da47_0 - numpy=1.23.5=py311h62c7003_0 - opencv=4.7.0=py311h6eed73b_2 - openh264=2.3.1=hf0c8a7f_2 - openjpeg=2.5.0=h13ac156_2 - openmotif=2.3.8=h6b0d8c2_4 - openssl=3.1.4=hd75f5a5_0 - p11-kit=0.24.1=h65f8906_0 - packaging=23.2=pyhd8ed1ab_0 - pandas=2.1.4=py311h1eadf79_0 - partd=1.4.1=pyhd8ed1ab_0 - patsy=0.5.6=pyhd8ed1ab_0 - pcre2=10.40=h1c4e4bc_0 - pep8=1.7.1=py_0 - pillow=9.5.0=py311h7cb0e2d_1 - pip=23.3.2=pyhd8ed1ab_0 - pixman=0.43.0=h73e2aa4_0 - pluggy=1.3.0=pyhd8ed1ab_0 - poppler=23.03.0=hc777cac_1 - poppler-data=0.4.12=hd8ed1ab_0 - postgresql=15.2=hbea33b9_0 - proj=9.1.1=hf909084_2 - pthread-stubs=0.4=hc929b4f_1001 - py-opencv=4.7.0=py311h9bdd199_2 - pycodestyle=2.11.1=pyhd8ed1ab_0 - pyflakes=3.2.0=pyhd8ed1ab_0 - pyparsing=3.1.1=pyhd8ed1ab_0 - pyproj=3.5.0=py311hdc3c720_0 - pyshp=2.3.1=pyhd8ed1ab_0 - pysocks=1.7.1=pyha2e5f31_6 - pytest=7.4.4=pyhd8ed1ab_0 - pytest-console-scripts=1.4.1=pyhd8ed1ab_0 - pytest-cov=4.1.0=pyhd8ed1ab_0 - pytest-mock=3.12.0=pyhd8ed1ab_0 - python=3.11.6=h30d4d87_0_cpython - python-dateutil=2.8.2=pyhd8ed1ab_0 - python-tzdata=2023.4=pyhd8ed1ab_0 - python_abi=3.11=4_cp311 - pytz=2023.3.post1=pyhd8ed1ab_0 - pywavelets=1.4.1=py311h4a70a88_1 - pyyaml=6.0.1=py311h2725bcf_1 - rasterio=1.3.6=py311hc41c901_0 - readline=8.2=h9e318b2_1 - regex=2023.12.25=py311he705e18_0 - remotezip=0.12.2=pyhd8ed1ab_0 - requests=2.31.0=pyhd8ed1ab_0 - responses=0.24.1=pyhd8ed1ab_0 - rtree=1.1.0=py311hbc1f44b_0 - s3transfer=0.10.0=pyhd8ed1ab_0 - scikit-image=0.20.0=py311hd84f3f5_1 - scikit-learn=1.3.2=py311h66081b9_2 - scipy=1.11.4=py311he0bea55_0 - setuptools-scm=8.0.4=pyhd8ed1ab_0 - setuptools_scm=8.0.4=hd8ed1ab_0 - shapely=2.0.1=py311heb7bb94_1 - six=1.16.0=pyh6c4a22f_0 - snappy=1.1.10=h225ccf5_0 - snuggs=1.4.7=py_0 - sqlite=3.44.2=h7461747_0 - statsmodels=0.14.1=py311hc9a392d_0 - svt-av1=1.4.1=hf0c8a7f_0 - tabulate=0.9.0=pyhd8ed1ab_1 - tenacity=8.2.2=pyhd8ed1ab_0 - threadpoolctl=3.2.0=pyha21a80b_0 - tifffile=2020.6.3=py_0 - tiledb=2.13.2=h8b9cbf0_0 - tk=8.6.13=h1abcd95_1 - toml=0.10.2=pyhd8ed1ab_0 - tomli=2.0.1=pyhd8ed1ab_0 - toolz=0.12.0=pyhd8ed1ab_0 - tqdm=4.66.1=pyhd8ed1ab_0 - types-pyyaml=6.0.12.12=pyhd8ed1ab_0 - typing-extensions=4.9.0=hd8ed1ab_0 - typing_extensions=4.9.0=pyha770c72_0 - tzcode=2023d=h10d778d_0 - tzdata=2023d=h0c530f3_0 - tzlocal=5.2=py311h6eed73b_0 - uriparser=0.9.7=hf0c8a7f_1 - urllib3=1.26.18=pyhd8ed1ab_0 - x264=1!164.3095=h775f41a_2 - x265=3.5=hbb4e6a2_3 - xerces-c=3.2.4=h2007e90_1 - xorg-kbproto=1.0.7=h35c211d_1002 - xorg-libice=1.0.10=h0d85af4_0 - xorg-libsm=1.2.3=h0d85af4_1000 - xorg-libx11=1.6.12=h9a43bea_0 - xorg-libxau=1.0.11=h0dc2134_0 - xorg-libxdmcp=1.1.3=h35c211d_0 - xorg-libxext=1.3.4=h01d97ff_0 - xorg-libxft=2.3.4=hf9f1598_1 - xorg-libxmu=1.1.3=h0b31af3_0 - xorg-libxp=1.0.3=h10d778d_1002 - xorg-libxrender=0.9.10=h01d97ff_1002 - xorg-libxt=1.1.5=h01d97ff_1003 - xorg-renderproto=0.11.1=h0d85af4_1002 - xorg-xextproto=7.3.0=hb7f2c08_1003 - xorg-xproto=7.0.31=h35c211d_1007 - xyzservices=2023.10.1=pyhd8ed1ab_0 - xz=5.2.6=h775f41a_0 - yaml=0.2.5=h0d85af4_2 - zipp=3.17.0=pyhd8ed1ab_0 - zlib=1.2.13=h8a1eda9_5 - zstd=1.5.5=h829000d_0 - pip: - hyp3-isce2==0.9.2 - setuptools==67.6.1 - wheel==0.40.0 ```
forrestfwilliams commented 6 months ago

@scottyhq what MacOS are you running? I'm on Sonoma 14.1 and can't recreate the issue.

scottyhq commented 6 months ago

Thanks for looking into it @forrestfwilliams, after some frustrating debugging I realized I had GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR set in my terminal (often these days since working mostly with URLs). And Because the ISCE driver seems to require sidecar files even to open new files, gdal unceremoniously exits :(, thus the traceback!

jhkennedy commented 6 months ago

@scottyhq oh, that's handy to know! I set GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR often as well.