junchangju
commented
2 years ago I wrote a very simple script named "s2nonempty.sh" which returns 1 if the SAFE is non-empty and 0 if empty. So if the return is 0, you can exit early. It takes around 8 seconds to report if the SAFE is non-empty and almost no time if empty but of course the latter occurs only rarely. I only test only on one band, B06, which should be enough, but testing all bands would take up to 2 minutes, which is probably not worthwhile? I'm sending you the script in email because attachment of Unix script here is not allowed.
@freitagb and @junchangju We discussed this on Slack but I'm including it here as an issue for tracking purposes.
I’ve been investigating some of the errors (less than 0.05%) that were failing trying to generate CMR metadata for the images. It appears that this occurs when the S2 granule has no valid data. It is fully processed but as there is no valid data the metadata generation fails when we attempt to create the valid data polygon https://github.com/NASA-IMPACT/hls-metadata/blob/master/metadata_creator/metadata_creator.py#L334. Here is an example S2 granule I downloaded that appears to have no valid data
s3://hls-sentinel-validation-scenes/cloud_free_google/S2B_MSIL1C_20220519T020649_N0400_R103_T53UPQ_20220519T040527.zipIs there a way we should initially detect these empty granules via S2 metadata information and exit early with a specific exit code to avoid the wasted processing time for Fmask and Lasrc rather than failing with a generic exit code 1 (which makes failure tracking difficult for these cases).