Multiple bands detected from S2B L1A S2COR

[andreabenedetti]

Hi,

I'm having some issues in trying to run correctly waterdetect. I tried to use it on a variety of imagery from Copernicus but it failed everytime.

I'm trying to run it on this specific tile S2B_MSIL2A_20210613T070619_N0300_R106_T40TEM_20210613T085847.SAFE, which should be compatible with the product S2_S2COR. However, when waterdetect runs, it returns this error:

Loading configuration file WaterDetect.ini
File WaterDetect.ini verified.
Folder /Volumes/ANSSD/testFolder/input_folder verified.
Folder /Volumes/ANSSD/testFolder/output_folder verified.
Starting WaterDetection version: 1.5.8
Opening image in folder /Volumes/ANSSD/testFolder/input_folder/S2B_MSIL2A_20210613T070619_N0300_R106_T40TEM_20210613T085847.SAFE
Retrieving bands for product S2_S2COR
70 were found:
Loading band: Blue
****** WARNING ********
More than one band _B02_10m. in image list. Skipping to the next image

Do you have any insight on how to fix it and correctly run waterdetect?

Thanks in advance!

[cordmaur]

Hi, It is strange because it is finding exactly the the double of bands in your image. That's why it is raising an exception. I have just tested here and take a look a the results.

Loading configuration file WaterDetect.ini
File WaterDetect.ini verified.
Folder d:/images/input/Input_S2COR verified.
Folder d:/temp verified.
Starting WaterDetection version: 1.5.8
Opening image in folder d:\images\input\Input_S2COR\S2A_MSIL2A_20180819T105031_N0208_R051_T31TCM_20180819T145346.SAFE
Retrieving bands for product S2_S2COR
35 were found:
Loading band: Blue
Loading band: Green
Loading band: Red
Loading band: Mir
Loading band: Mir2
Loading band: RedEdg1
Loading band: RedEdg2
Loading band: RedEdg3
Loading band: Nir
Loading band: Nir2

Please, check inside your img_folder/GRANULE/L2A_T40TEM_xxxxxxx/IMG_DATA You should have 3 folders:

R10m
R20m
R60m

Inside R10m you should have something like this...

Let me know if there are any inconsistencies.

Mauricio

[andreabenedetti]

It's weird, I tried with another image but the results were the same: 70 bands found, exactly double what should be expected. The folders structure is the same, and the R10m contains something similar to the one you used:

[andreabenedetti]

A small update: it also happens with images using the product S2_L1C. Maybe it's a problem of the source of the images? This is the error message:

(base) andrea@Andreas-MacBook-Pro WaterDetect-1.5.8 % python runWaterDetect.py -i /Volumes/ANSSD/testFolder/input_folder -o /Volumes/ANSSD/testFolder/output_folder -p S2_L1C
Loading configuration file WaterDetect.ini
File WaterDetect.ini verified.
Folder /Volumes/ANSSD/testFolder/input_folder verified.
Folder /Volumes/ANSSD/testFolder/output_folder verified.
Starting WaterDetection version: 1.5.8
Opening image in folder /Volumes/ANSSD/testFolder/input_folder/S2A_MSIL1C_20210602T064631_N0300_R020_T40TGP_20210602T093931.SAFE
Retrieving bands for product S2_L1C
28 were found:
Loading band: Blue
****** WARNING ********
More than one band _B02. in image list. Skipping to the next image

[cordmaur]

I have only seen such problem when we open the bands in a different software that creates auxiliary files with the name similar to the names of the bands. Can you share this image so I can take a look at it?

[andreabenedetti]

First image, using product S2_S2COR: https://scihub.copernicus.eu/dhus/odata/v1/Products('d841af68-b47e-4515-af3b-de234995c6b8')/$value Second image, using product S2_L1C: https://scihub.copernicus.eu/dhus/odata/v1/Products('4a8e5558-2c01-4fc8-a117-9a083ee0624b')/$value

[cordmaur]

Hi Andrea, I downloaded the images and ran the waterdetect. Here is the output:

The image is normal and the code is working. To check what is going on, I suggest you follow these commands on Jupyter:

This is the function I use to retrieve the bands in the product to filter them afterwards. This way, we can see everything that the code is finding in your drive. Let me know of the results.

PS: just for my statistics purposes, can you let me know where are you from? Country and University/Enterprise?

Regards,

[andreabenedetti]

It's still not working, but I think I'm understanding where the issue lies: the tiles I downloaded don't have a R30m folder, rather a R60m folder, which probably contains a different set of information than what waterdetect expects.

However I'm struggling to understand which kind of imagery I am supposed to use with this script, since I downloaded tiles from the Sentinel 2 satellite, of the type S2A and L2A.

Regarding my background: I'm a researcher and PhD candidate in design from Italy, specifically Politecnico di Milano at the Design Department!

[cordmaur]

Andrea,

I did a mistake on my previous posts. The folders inside IMG_DATA are:

R10m
R20m
R60m

I have ran my last test with the images downloaded from the link you provided, so they are exactly the same images. It may be something in the system or any previous processing you may have done. If could create a DWLoader as I showed, it will be easier to debug the problem.

Regards, Mauricio

[andreabenedetti]

Hi Mauricio,

problem is, the DWLoader doesn't recognize the images as valid S2COR imagery! I didn't manipulate them in any way, I just downloaded them straight from Copernicus. Here is the Jupyter notebook:

And here the image folder:

Thank you for your patience! I'm fairly new to this kind of image processing and analysis :)

[cordmaur]

Hi Andrea, No problem. We need to find out what's going on. But it's weird as the first problem is that it was finding the double of bands, and now it finds none. I will ask you to follow these commands to try to debug the issue (if you are running linux, you change dir for ls):

And then, we will take a look inside the image.

Finally we will check if the configuration is good for opening it. If everything matches, it should work at this point.

[andreabenedetti]

Hi Mauricio,

thanks! I'm running the code. Everything is the same up to the last step. Here the breakdown.

Loading folder

Looking at the images

Configuration check

[andreabenedetti]

Just for reference, the image I'm trying to analyze is this one, so I think there should definitely be some water detected! :)

[cordmaur]

It's strange because the first error you've sent me was because it was finding the double of bands, and now it finds no bands. And this image is different from the one you asked me to download, but that's not the issue.

We have to debug it. I don't know if you are familiar with, but let's the basics.

Try to run loader.get_bands_files() and wait for the error. In the next cell you type: %debugand shift+enter

inside the debuger you will type the following commands (one at a time) and check the results:

> self.bands_path

> self.product_dict

> [f for f in self.bands_path.rglob('*.jp2')]

Here is an example (note that I changed the product to S2_THEIA to force an error at the same point)

If it does not return anything, try the rglob('*') to "see" what the class is seeing.

This is the image you sent me the link, already processed.

Just be aware that it has a lot of sun glint (you can check it on the PDF report also) and you may have to adjust the configuration depending on the objectives (include x exclude the glint, for example).

[andreabenedetti]

Hi Mauricio,

thanks for the patience! I ran the code you wrote, however when running [f for f in self.bands_path.rglob('*.jp2')] the result is an empty array, even when running [f for f in self.bands_path.rglob('*')].

Here the logs:

[cordmaur]

Andrea. You have to run the tests mentioned here (https://github.com/cordmaur/WaterDetect/issues/11#issuecomment-863852114) again. And then do the debug when the error occurs. Note that the Path shown in the debug is “duplicated” and it does not match the path shown in the previous post.

[cordmaur]

Hi @andreabenedetti , could you succeed to solve the problem?

[haozhen315]

I have downloaded the imgaes with the links provided in this post and just fresh downloaded waterdetect with running: python runWaterDetect.py -i /Volumes/SanDisk/DownloadSanDisk/Test_S2COR -o /Volumes/SanDisk/DownloadSanDisk/Test_S2COR_Out -p S2_S2COR

and I have got the result, so I believe there is no problem with author's code. test on Mac

[cordmaur]

Thanks @haozhen315