aria-tools / ARIA-tools

Tools for exploiting ARIA standard products
Apache License 2.0
99 stars 36 forks source link

[BUG] bug when passing `NUM_THREADS` options through gdalWarp #422

Closed sssangha closed 5 months ago

sssangha commented 5 months ago

Describe the bug With the latest increment in gdal (3.9.0), the NUM_THREADS variable now raises an error when passed through gdalWarp.

To Reproduce Steps to reproduce the behavior:

  1. Download: ariaDownload.py --num_threads 16 -b '26.34 27.86 36.767 39.874' -s 20150401 -e 20150501 -t 14 -o Download

  2. Command used: ariaTSsetup.py -f "products/*.nc" -b '26.34 27.86 36.767 39.874' --verbose -w test_dir_all_redo -l all --log-level debug

  3. Error Output:

    (ARIA-tools_dev) [ssangha@trappist issue_420_ariatssetup]$ ariaTSsetup.py -f "products/*.nc" -b '33.6 34.3 -118.5 -117.5' --verbose -w test_dir_extract -l amplitude --log-level debug
    2024-06-17 14:43:44,679 - ariaTSsetup.py - INFO - ARIAtools version: 1.1.8
    2024-06-17 14:43:44,679 - ariaTSsetup.py - INFO - Time-series Preparation Function
    2024-06-17 14:43:44,679 - ariaTSsetup.py - INFO - Thread count specified for gdal multiprocessing = 2
    2024-06-17 14:43:44,679 - ariaTSsetup.py - INFO - Building ARIA product instance
    2024-06-17 14:43:44,781 - ARIAtools.product - INFO - Shapefile test_dir_extract/user_bbox.json created for input user bounds
    2024-06-17 14:43:45,100 - ARIAtools.product - INFO - All (8) GUNW products meet spatial bbox criteria.
    2024-06-17 14:43:45,101 - ARIAtools.product - INFO - Group GUNW products into spatiotemporally continuous interferograms.
    2024-06-17 14:43:45,103 - ARIAtools.product - INFO - All (3) interferograms are spatially continuous.
    2024-06-17 14:43:45,103 - ariaTSsetup.py - INFO - Extracting and merging product bounding boxes
    gdal_warp_kwargs {'format': 'MEM', 'multithread': True, 'dstSRS': 'EPSG:4326', 'options': ['NUM_THREADS=2']}
    Traceback (most recent call last):
    File "/u/trappist-r0/ssangha/conda_installation/miniforge/miniforge/ARIA-tools_dev/tools/bin/ariaTSsetup.py", line 632, in <module>
    main()
    File "/u/trappist-r0/ssangha/conda_installation/miniforge/miniforge/ARIA-tools_dev/tools/bin/ariaTSsetup.py", line 423, in main
    ARIAtools.extractProduct.merged_productbbox(
    File "/u/trappist-r0/ssangha/conda_installation/miniforge/miniforge/ARIA-tools_dev/tools/ARIAtools/extractProduct.py", line 486, in merged_productbbox
    warp_options = osgeo.gdal.WarpOptions(**gdal_warp_kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/u/trappist-r0/ssangha/conda_installation/miniforge/miniforge/envs/ARIA-tools_dev/lib/python3.12/site-packages/osgeo/gdal.py", line 1115, in WarpOptions
    return (GDALWarpAppOptions(new_options), callback, callback_data)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/u/trappist-r0/ssangha/conda_installation/miniforge/miniforge/envs/ARIA-tools_dev/lib/python3.12/site-packages/osgeo/gdal.py", line 8987, in __init__
    _gdal.GDALWarpAppOptions_swiginit(self, _gdal.new_GDALWarpAppOptions(*args))

Desktop (please complete the following information):

Additional context I have a working patch, just raising this issue ticket to record and reference it.