search

PatrickTUM / UnCRtainTS

https://patricktum.github.io/cloud_removal/
47 stars 3 forks source link

Some questions about the SEN12MSCRTS dataset class #8

Open Ly403 opened 5 months ago

Ly403 commented 5 months ago

Hi. This is truly a great paper! And I have encountered some issues when using the data/dataLoader.py/SEN12MSCRTS class for my own cloud removal model. First sorry for my poor English.

Here are my questions:

  1. I see that you have provided some pre-computed cloud coverage statistics on the pCloud. So, I downloaded the generic_3_train_all_s2cloudless_mask.npy, the generic_3_val_all_s2cloudless_mask.npy and the generic_3_test_all_s2cloudless_mask.npy as I set the input_t=3, region='all', sampler='fixed' and sample_type='cloudy_cloudfree' in my SEN12MSCRTS class. But by debugging, I have discovered that, in most cases, in the same area (that's to say the same pdx in the dataset), among all 30 images, the cloud coverage of more than one image is 0. The SEN12MSCRTS class selects the image with the lowest coverage in the fixed_sampler method as the target cloudless image. But under this circumstance, do we still need to conduct cloud removal since we already have another cloudless(coverage close to 0) image in the input image list?

  2. I have also observed that even when two or more images in the same area out of the 30 images have zero coverage, they do not look the same. I found this by drawing the RGB channels of some images. For example, the two images below are from the same area, but they have obvious differences. Does that mean we cannot directly view the cloud removal task as an inpainting task?

    Fig.1. Fig.2.

  3. Since we have more than one image out of the 30 images with zero coverage, does it matter which image we choose as the target cloudless image? That is to say, does the way we choose the target image affect the performance of our models?

  4. In a few cases, I have noticed that the target cloudless image appears to have clouds by observing the RGB channels with my eyes. For example, the four images below are fetched from the target S2 images using the __getitem__ method in the SEN12MSCRTS class and displayed as RGB images by selecting the RGB channels ([3, 2, 1]) from the original tif images. Actually, they do not seem to be cloudless. But this thing doesn't happen very often. Is this because there was an error in my code, or did the S2PixelCloudDetector fail to detect the cloud? Or for some other reason?

    Fig.3.

Apologize again for my poor English. Sincerely looking forward to your reply. Many thanks to you!

Ly403 commented 5 months ago

Sorry for the fourth question cause that's my fault. I used the _generic_3_train_all_s2cloudlessmask.npy, the _generic_3_val_all_s2cloudlessmask.npy and the _generic_3_test_all_s2cloudlessmask.npy, but I set the region!=all. That's why the cloudy images were chosen as the target images. But the rest three questions still puzzle me.

Again apologize for my fault.

someshfengde commented 2 months ago

@Ly403 have you managed to run the model? I was able to load the model but wasn't able to do inference because the image path formats are nto well defined?