error in GDAL call - Githubissues

lucadelu commented 8 months ago

Continuing testing I found this error, the config file is attached. I'm running it with docker

QAI will not be saved
FORCE temp data will not be removed
Downloaded Satellite data will not be removed
Found 8 matching scenes of type S2_MSI_L1C, starting download...
Downloaded products:   0%|                       | 0/8 [00:00<?, ?product/s]S2A_MSIL1C_20200710T102031_N0500_R065_T32TLP_20230322T012603: 0.00B [00:00,
S2A_MSIL1C_20200710T102031_N0500_R065_T32TLP_20230322T012603: 0file [00:00,
S2A_MSIL1C_20200710T102031_N0500_R065_T32TLP_20230322T012603: 100%|█| 1/1 [0Downloaded products:  12%|█▉             | 1/8 [00:00<00:02,  2.79product/s]S2B_MSIL1C_20200705T101559_N0209_R065_T32TLP_20200705T131303: 0.00B [00:00,
S2B_MSIL1C_20200705T101559_N0209_R065_T32TLP_20200705T131303: 0file [00:00,
S2B_MSIL1C_20200705T101559_N0209_R065_T32TLP_20200705T131303: 100%|█| 1/1 [0
S2A_MSIL1C_20200703T103031_N0500_R108_T32TLP_20230613T124137: 0.00B [00:00,
S2A_MSIL1C_20200703T103031_N0500_R108_T32TLP_20230613T124137: 0file [00:00,
S2A_MSIL1C_20200703T103031_N0500_R108_T32TLP_20230613T124137: 100%|█| 1/1 [0
S2B_MSIL1C_20200708T102559_N0500_R108_T32TLP_20230528T041416: 0.00B [00:00,
S2B_MSIL1C_20200708T102559_N0500_R108_T32TLP_20230528T041416: 0file [00:00,
S2B_MSIL1C_20200708T102559_N0500_R108_T32TLP_20230528T041416: 100%|█| 1/1 [0
S2A_MSIL1C_20200710T102031_N0209_R065_T32TLP_20200710T121947: 0.00B [00:00,
S2A_MSIL1C_20200710T102031_N0209_R065_T32TLP_20200710T121947: 0file [00:00,
S2A_MSIL1C_20200710T102031_N0209_R065_T32TLP_20200710T121947: 100%|█| 1/1 [0
S2A_MSIL1C_20200703T103031_N0209_R108_T32TLP_20200703T124229: 0.00B [00:00,
S2A_MSIL1C_20200703T103031_N0209_R108_T32TLP_20200703T124229: 0file [00:00,
S2A_MSIL1C_20200703T103031_N0209_R108_T32TLP_20200703T124229: 100%|█| 1/1 [0
S2B_MSIL1C_20200705T101559_N0500_R065_T32TLP_20230530T084300: 0.00B [00:00,
S2B_MSIL1C_20200705T101559_N0500_R065_T32TLP_20230530T084300: 0file [00:00,
S2B_MSIL1C_20200705T101559_N0500_R065_T32TLP_20230530T084300: 100%|█| 1/1 [0
S2B_MSIL1C_20200708T102559_N0209_R108_T32TLP_20200708T124214: 0.00B [00:00,
S2B_MSIL1C_20200708T102559_N0209_R108_T32TLP_20200708T124214: 0file [00:00,
S2B_MSIL1C_20200708T102559_N0209_R108_T32TLP_20200708T124214: 100%|█| 1/1 [0Downloaded products: 100%|███████████████| 8/8 [00:00<00:00, 20.97product/s]Product type 'LANDSAT_C2L1' is not available with provider 'cop_dataspace'. Searching it on provider 'usgs' instead.
Found 4 matching scenes of type LANDSAT_C2L1, starting download...
LC08_L1TP_195029_20200710_20200912_02_T1: 100%|█| 1/1 [00:00<00:00, 5059.47fLC08_L1TP_194029_20200703_20200913_02_T1: 100%|█| 1/1 [00:00<00:00, 4568.96fLC08_L1TP_195030_20200710_20200912_02_T1: 100%|█| 1/1 [00:00<00:00, 3008.83fLC08_L1TP_194030_20200703_20200913_02_T1: 100%|█| 1/1 [00:00<00:00, 1060.24fDownloaded products: 100%|██████████████| 4/4 [11:29<00:00, 172.36s/product]Setting up FORCE processing...
Downloading Water Vapor Database to /data/wvdb/dir/...
Water Vapor Database exists already in /data/wvdb/dir/, skipping...
Running FORCE Level-2 Processing...
14 images enqueued. Start processing with 1 CPUs

Computers / CPU cores / Max jobs to run
1:local / 8 / 1

Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
ETA: 1216s Left: 1 AVG: 1216.00s  local:1/13/100%/1257.1s Couldn't open the directory: Not a directory
Couldn't open the directory: Not a directory
Couldn't open the directory: Not a directory
ETA: 0s Left: 0 AVG: 1129.14s  local:0/14/100%/1167.3s
/home/docker/.local/lib/python3.10/site-packages/sadasadam/force.py:47: UserWarning: Directory /data/output/ already exists, skipping...
  warnings.warn(f"Directory {directory} already exists, skipping...")
FORCE log files copied to /data/output/
Creation of same day mosaics...
Creation of same day mosaics finished
Postprocessing to clear sky mosaics...
/home/docker/.local/lib/python3.10/site-packages/sadasadam/force.py:47: UserWarning: Directory /data/output/ already exists, skipping...
  warnings.warn(f"Directory {directory} already exists, skipping...")
ERROR 1: Error: Computed -srcwin 3765 6388 -730 1503 has negative width and/or height.
ERROR 1: Error: Computed -srcwin 3765 6388 -730 1503 has negative width and/or height.
ERROR 1: Error: Computed -srcwin 11296 4166 -2191 4510 has negative width and/or height.
ERROR 1: Error: Computed -srcwin 11296 4166 -2191 4510 has negative width and/or height.
ERROR 1: Error: Computed -srcwin 11296 4166 -2191 4510 has negative width and/or height.
ERROR 1: Error: Computed -srcwin 11296 4166 -2191 4510 has negative width and/or height.
ERROR 1: Error: Computed -srcwin 11296 4166 -2191 4510 has negative width and/or height.
ERROR 1: Error: Computed -srcwin 11296 4166 -2191 4510 has negative width and/or height.
ERROR 1: Error: Computed -srcwin 7765 6388 -730 1503 has negative width and/or height.
ERROR 1: Error: Computed -srcwin 7765 6388 -730 1503 has negative width and/or height.
ERROR 1: Error: Computed -srcwin 11296 4166 -2191 4510 has negative width and/or height.
ERROR 1: Error: Computed -srcwin 11296 4166 -2191 4510 has negative width and/or height.
ERROR 1: Error: Computed -srcwin 11296 4166 -2191 4510 has negative width and/or height.
ERROR 1: Error: Computed -srcwin 11296 4166 -2191 4510 has negative width and/or height.
ERROR 4: /data/temp_force/force_dir_20240117_091114/level2/mosaic/20200703_LEVEL2_LND08_clearsky.tif: No such file or directory
Traceback (most recent call last):
  File "/home/docker/.local/bin/sadasadam", line 8, in <module>
    sys.exit(main())
  File "/home/docker/.local/lib/python3.10/site-packages/sadasadam/cli.py", line 250, in main
    force_proc.postprocess(
  File "/home/docker/.local/lib/python3.10/site-packages/sadasadam/force.py", line 434, in postprocess
    band_clearsky = ds_clearsky.GetRasterBand(1)
AttributeError: 'NoneType' object has no attribute 'GetRasterBand'
docker@cdaf0a5bd43e:~/src/sadasadam$

The error seems related to GDAL and could be this line

neteler commented 8 months ago

May this be the problem?

Couldn't open the directory: Not a directory
Couldn't open the directory: Not a directory
Couldn't open the directory: Not a directory

lucadelu commented 8 months ago

May this be the problem?

Couldn't open the directory: Not a directory
Couldn't open the directory: Not a directory
Couldn't open the directory: Not a directory

I don't think so, I think this error could be related to the last ERROR 4: No such file

The ERROR lines are 14 as the same number of images and in the temp files we have the mosaic VRT from FORCE only last two steps are needed.

@griembauer @neteler could we have a call next week? to make some debug together... we will download one month of data before the call and after that we try to analyse a couple of images....

griembauer commented 8 months ago

I haven't really looked into it yet, but might it be this from the config file:

# type=float, help='Eastern boundary of AOI in decimal degrees'
east: 7.45

# type=float, help='Western boundary of AOI in decimal degrees'
west: 7.713

--> Are west and east swapped? West should not be a larger number than east --> west should be in decimal degrees E, so using west: 7.713 is not understood as -7.713 degree East if this was the goal here. This might be confusing so we should add an explanation to the comments in the config file and a first check whether the coordinates make sense in cli.py.

If that's the issue, it is curious that the error occurs so late in the processing and eodag download seems not to be bothered by it.

Update: Sanity check added in https://github.com/mundialis/sadasadam/pull/14

lucadelu commented 8 months ago

It seems west and east are swapped, we are going to fix this and run again. I let you know as soon as possible. Yes, strange that eodag doesn't bothered about this

mundialis / sadasadam

error in GDAL call #13