sertit / eoreader

Remote-sensing opensource python library reading optical and SAR sensors, loading and stacking bands, clouds, DEM and spectral indices in a sensor-agnostic way.
https://eoreader.readthedocs.io/en/latest/
Apache License 2.0
278 stars 22 forks source link

The alignment between EOReader and SNAP #65

Closed TK12331 closed 1 year ago

TK12331 commented 1 year ago

I process Sentinel-1 data with SNAP before, and now I want to align the processing results between EOReader and SNAP. I process GRD product using EOReader with step: Apply-Orbit-File -> ThermalNoiseRemoval -> Remove-GRD-Border-Noise -> Calibration-> Terrain-Correction. (for comparing the value of pixel, I remove step LinearToFromdB currently.

The name of the GRD product I use is S1A_IW_GRDH_1SDV_20200308T121318_20200308T121343_031584_03A39B_5C41.zip, whose image shape is 25847 x 16731.

However, I found some problems as followes:

  1. Comparation betweenCalibration product The value of tif file I generated from SNAP is 0-255(float64,but all the decimal value is 0), while the value of tif file I generated from EOReader is 0-123.4. Besides, the scale value between corresponding pixels is not constant.

  2. Comparation between Terrain-Correction product The value problem is as same as 1. The shape of tif file I generated from SNAP is 31668 x 21478, while the shape of tif file I generated from EOReader is 14372 x 10790.

Why they aren't alignment? Or am I missing something?

remi-braun commented 1 year ago

What is the resolution of your SNAP product ? By default EOReader sets a 20m resolution to Sentinel-1 IW GRD products as stated and explained here. Maybe SNAP sets another default resolution ?

It may also come from another CRS reprojection. EOReader reprojects everything in UTM when SNAP does not if I'm not mistaken.

You can try a gdalinfo on the both files to check their respective resolution and CRS.

If you want to modify the graphs, just follow this section.

Moreover, for discussions could you post in the discussions section please ? I would like to keep this section for bugs and enhancements πŸ˜„ And don't hesitate to look through the documentation before asking here, a lot of what you ask is already there πŸ˜‰

TK12331 commented 1 year ago

OK, I will post it in the discussion. Thank you!πŸ˜‰

remi-braun commented 1 year ago

Your comments triggered an intense discussion here at SERTIT (thanks for this πŸ˜„) ➑️ Should we load bands to the resolution rg*az or to the pixel spacing (i.e. with these given values)

Before today, we settled on orthorectifying data to the "real" resolution (ie rg*az), in order to save space (images are smaller) and by knowing that objects under this minimum mapping unit wouldn't be resolved.

However, we realised that this may disorient the user as this isn't the common habits. As for 0.18.0+, provided pixel spacing will be used as default resolution for SAR data, i.e. 10m for S1 GRD IW πŸ˜„

remi-braun commented 1 year ago

Sorry for my afterthought, but we have to follow CEMS resolutions defines in the Data Access Portfolio (2014-2022, section 6.2) as EOReader is first built for CEMS activations πŸ˜…

I will then only update resolutions for products that are not in this document. (you see that S1 GRD IW are at 20m resolution there)

remi-braun commented 1 year ago

Does this answer to your question ?

TK12331 commented 1 year ago

Oh, thanks for your reply! which makes me understand more about SAR data processing. You mentioned about SERTIT, what's that?

However, I still can not align the output tiff with SNAP and EOReader: As mentioned before, the value of tif file I generated from SNAP using export tiff is 0-255(float64,but all the decimal value is 0), while the value of tif file I generated from EOReader is 0-123.4(float64). Besides, the scale value between corresponding pixels is not constant.

For afterward image processing, I want to align the data produced by SNAP and EOReader, so I want to make sure the distribution of my SAR data is consistent.

How can I make it?

Here are some comparation of tiff value, which is produced by SNAP(left) and EOReader(right), resectively. The location of value is around [3000,3000] ( the shape of tiff is 25847 x 16731).

image

remi-braun commented 1 year ago

In must say, it is really unlikely that your SAR data stays in integer Could you post the two graphs you used for producing these rasters ? πŸ˜€

TK12331 commented 1 year ago

It seems that export tiff in SNAP do some normalization...

Now I use Geo-Tiff from SNAP to make alignment, the XML for SNAP and EOReader are added in attachment. graph.zip

TK12331 commented 1 year ago

I use GDAL to read tiff file, the middle name ge means the geo-tiff from SNAP, while `eo means the tiff from EOReader. The band I select is VH image

Further test: image image

remi-braun commented 1 year ago

It seems that export tiff in SNAP do some normalization...

Now I use Geo-Tiff from SNAP to make alignment, the XML for SNAP and EOReader are added in attachment. graph.zip

I see that the two graphs are different. In EOReader you left the Thermal Noise Removal step.

Moreover other fields are different, such as in auxFile in Calibration.

Could you test this again with exact same graphs ?

TK12331 commented 1 year ago

OK, I test the same graphs right now.

The thermal Noise Removal is done in original product, SNAP will raised an error, would EOReader continue to process this step?

Besides, u can check the comparison result I update above, it seems that they are very close ... The mean of their difference for each pixel is just 0.003, and the max of their difference is just 0.019.

From your opinion, are they alignment?

remi-braun commented 1 year ago

From what I recollect, your comparison is flawed as the graphs are no exactly the same. (or I didn't understand this sentence:

The thermal Noise Removal is done in original product, SNAP will raised an error, would EOReader continue to process this step?

)

Moreover, as I am using directly SNAP without any modification, I wouldn't understand discrepancies (and I wouldn't know how to fix them).

Eventually, these discrepancies, as you underlined, are thin. So I personally wouldn't bother to check it more, for the following reason : ➑️ You will add a terrain correction step after that that will resample everything, mixing speckle noise from every neighbouring pixels, so the absolute pixel values are not really making sense. Not to mention the despeckling step that will do that a lot more.

TK12331 commented 1 year ago

Thanks! I have alligned the Calibration product now.

remi-braun commented 1 year ago

Ok, I am closing this ;)