search

Open-EO / openeo-geopyspark-driver

OpenEO driver for GeoPySpark (Geotrellis)
Apache License 2.0
26 stars 4 forks source link

orfeo thermal noise removal introduces nodata #293

Open jdries opened 1 year ago

jdries commented 1 year ago

snap procedure is to set thermal noise to a very low value instead We temporarily solved it with a workaround in user code: if_(is_nodata(x[0]),1,30.0 + 10.0 * x[0].log(base=10))

jdries commented 1 year ago

we can try setting -outputs.default to some value other than 0, hoping that denoising than still uses 0

JeroenVerstraelen commented 1 year ago

@jdries Which value other than 0 would be best for this?

jdries commented 1 year ago

not sure what datatype is used? Maybe set a high value, like 10000?

JeroenVerstraelen commented 1 year ago

It looks like the result dtype is always float32.

jdries commented 1 year ago

output is backscatter, which is usually a small decimal number, so any high floating point number should work. Do you get output already?

JeroenVerstraelen commented 1 year ago

Yes, I am getting output. I'm just working on a better example test. I've been using test_s1backscatter_orfeo.py::test_creodias_s1_backscatter for now. The noise removal is a bit strange because it both sets a good amount of noise pixels to NoData (=0), but also changes the [min, max] of the output from [0.000706709, 4.4512] to [1.20978e-08, 4.44767] which I assume is what you mean with "thermal noise is set to a very low value instead".

I found the detailed version of the outputs.default parameter description in the cli, because at first I was not sure what default pixel actually meant:

        -outputs.default         <float>          Default pixel value  (optional, off by default, default value is 0)
                                                  Default value to write when outside of input image

Am I correct that the issue is that noise_removal sets pixels close to 0, so we want pixels outside of the input image to be different from 0 so we can distinguish them?

jdries commented 1 year ago

yes, and I'm also hoping that noise removal does not use the nodata value set by outputs.default at all, so that is to be tested.

JeroenVerstraelen commented 1 year ago

During testing we found out that there is a bug in the orfeo toolbox. In the output, only part of the no-input pixels will have the value 10000.0, the other part will still be 0.

This issue has been logged at orfeo: https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2329

Currently waiting for a response.

JeroenVerstraelen commented 1 year ago

Reverted the usage of outputs.default until the orfeo issue is fixed.

JeroenVerstraelen commented 1 year ago

Response on https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2329 Waiting for the fix I recommend you to call ManageNoData after the orthorectification, which is made for your use case.

soxofaan commented 5 months ago

hmm https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2329 has been closed now:

Closed because author of issue didn't provide needed information