Force level2: high level of cloud cover despite low level

[kacpermisiek]

Hi,

I have encountered an issue while running the force-l2ps script, specifically with the cloud coverage calculation at the start of the process. The script is reporting unusually high cloud coverage for certain products, even though the metadata for these products indicates low cloud coverage. Additionally, the quicklook images of these products also show low cloud coverage, which further contradicts the script’s output.

I am downloading products from explore.creodias.eu, here is example product that i am having problems with: https://explore.creodias.eu/search/details/S2B_MSIL1C_20240725T001119_N0511_R073_T55JEK_20240725T013419.SAFE?catalogueId=creodias

As you can see, the cloud cover is equal to 0:

Another example: https://explore.creodias.eu/search/details/S2B_MSIL1C_20221005T105819_N0400_R094_T31UFA_20221005T114128.SAFE?catalogueId=creodias Product with 22% of cloud coverage, but calculated value is 75.05%

Expected behavior Calculated cloud coverage should be lower, and input product should be able to be processed.

Parameterization Command:

/usr/local/bin/force-l2ps /mnt/vol/input/S2B_MSIL1C_20240725T001119_N0511_R073_T55JEK_20240725T013419.SAFE /force3-l2ps-params.prm  

Stdout:

dc:   0.00%. wc:   -nan%. sc:   -nan%. cc: 100.00%. Skip. Processing time: 00 mins 46 secs

Params file:

++PARAM_LEVEL2_START++

# INPUT/OUTPUT DIRECTORIES
# ------------------------------------------------------------------------
FILE_QUEUE = NULL
DIR_LEVEL2 = /mnt/vol/tmp
DIR_LOG = /mnt/vol/logs
DIR_TEMP = /mnt/vol/tmp
DIR_PROVENANCE = /mnt/vol/tmp

# DIGITAL ELEVATION MODEL
# ------------------------------------------------------------------------
FILE_DEM = ./dem-force.tif
DEM_NODATA = -32767

# DATA CUBES
# ------------------------------------------------------------------------
DO_REPROJ = FALSE
DO_TILE = FALSE
FILE_TILE = NULL
TILE_SIZE = 30000
BLOCK_SIZE = 3000
RESOLUTION_LANDSAT = 30
RESOLUTION_SENTINEL2 = 10
ORIGIN_LON = -25
ORIGIN_LAT = 60
PROJECTION = GLANCE7
RESAMPLING = CC

# RADIOMETRIC CORRECTION OPTIONS
# ------------------------------------------------------------------------
DO_ATMO = TRUE
DO_TOPO = TRUE
DO_BRDF = TRUE
ADJACENCY_EFFECT = TRUE
MULTI_SCATTERING = TRUE

# WATER VAPOR CORRECTION OPTIONS
# ------------------------------------------------------------------------
DIR_WVPLUT = NULL
WATER_VAPOR = NULL

# AEROSOL OPTICAL DEPTH OPTIONS
# ------------------------------------------------------------------------
DO_AOD  = TRUE
DIR_AOD  = NULL

# CLOUD DETECTION OPTIONS
# ------------------------------------------------------------------------
ERASE_CLOUDS = FALSE
MAX_CLOUD_COVER_FRAME = 75
MAX_CLOUD_COVER_TILE  = 75
CLOUD_BUFFER  = 300
SHADOW_BUFFER = 90
SNOW_BUFFER   = 30
CLOUD_THRESHOLD  = 0.225
SHADOW_THRESHOLD = 0.02
CIRRUS_BUFFER = 0

# RESOLUTION MERGING
# ------------------------------------------------------------------------
# This parameter defines the method used for improving the spatial reso-
# lution of Sentinel-2’s 20 m bands to 10 m. Pixels flagged as cloud or
# shadow will be skipped. Following methods are available: IMPROPHE uses
# the ImproPhe code in a spectral-only setup; REGRESSION uses a multi-
# parameter regression (results are expected to be best, but processing
# time is significant); STARFM uses a spectral-only setup of the Spatial
# and Temporal Adaptive Reflectance Fusion Model (prediction artifacts
# may occur between land cover boundaries); NONE disables resolution merge;
# in this case, 20m bands are quadrupled.
# Type: Character. Valid values: {IMPROPHE,REGRESSION,STARFM,NONE}
RES_MERGE = REGRESSION

# CO-REGISTRATION OPTIONS
# ------------------------------------------------------------------------
# This parameter only applies for Sentinel-2 data. This parameter defines
# the path to a directory that contains monthly Landsat NIR base images.
# If given, a co-registration is attempted. If it fails (no tie points),
# the image won't be processed.
# Type: full directory path
DIR_COREG_BASE = NULL
COREG_BASE_NODATA = -32767

# MISCELLANEOUS OPTIONS
# ------------------------------------------------------------------------
IMPULSE_NOISE = TRUE
BUFFER_NODATA = FALSE

# TIER LEVEL
# ------------------------------------------------------------------------
TIER = 1

# PARALLEL PROCESSING
# ------------------------------------------------------------------------
# Multiprocessing options (NPROC, DELAY) only apply when using the batch
# utility force-level2. They are not used by the core function force-l2ps.
# ------------------------------------------------------------------------
NPROC = 32
NTHREAD = 2
PARALLEL_READS = FALSE
DELAY = 3
TIMEOUT_ZIP = 30

# OUTPUT OPTIONS
# ------------------------------------------------------------------------
OUTPUT_FORMAT = GTiff
OUTPUT_DST = TRUE
OUTPUT_AOD = TRUE
OUTPUT_WVP = TRUE
OUTPUT_VZN = TRUE
OUTPUT_HOT = TRUE
OUTPUT_OVV = TRUE
FILE_OUTPUT_OPTIONS = NULL

++PARAM_LEVEL2_END++

Setup

• FORCE version: 3.7.12
• bare metal Linux server or virtual machine? Virtual machine
• CPU: 4
• RAM: 10Gi

[davidfrantz]

Hi @kacpermisiek

I just got back from vacation. Thank you for your patience.

I don't see any issues on my end. I tested the following image (downloaded from CDSE since I don't have a Creodias account):

S2B_MSIL1C_20240725T001119_N0511_R073_T55JEK_20240725T013419.SAFE

This is what I get: dc: 22.29%. wc: 0.01%. sc: 0.01%. cc: 0.27%. AOD: 0.1035. # of targets: 0/3. 1 product(s) written. Success! Processing time: 01 mins 06 secs

Your output was: dc: 0.00%. wc: -nan%. sc: -nan%. cc: 100.00%. Skip. Processing time: 00 mins 46 secs

The issue here is not the cloud coverage, but the data coverage (first number). Usually, this happens when the provided DEM doesn't cover the image extent.

Cheers, David