Test results

[Lithsun]

Hi Lanha,

I test your trained model with RMSE,SRE,SAM, UIQ metrics. But the UIQ and SAM values are not same as the results in your paper. I use the UIQ and SAM functions that are provied by sewar (one python package).

[lanha]

Are the RMSE and SRE metrics the same?

[Lithsun]

RMSE and SRE are same . SAM and UIQ are higher.

Those metrics functions are provided by one python package, which is named sewar. (You can check the document here https://sewar.readthedocs.io/en/latest/)

Besides, two test images have black area where the pixel values are 0. I'd like to know how you deal with it when calculating SAM and UIQ.

On 09/16/2019 17:29, Charis Lanaras wrote:

Are the RMSE and SRE metrics the same?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[Lithsun]

Are the RMSE and SRE metrics the same?

RMSE and SRE are same . SAM and UIQ are higher.

Those metrics functions are provided by one python package, which is named sewar. (You can check the document here https://sewar.readthedocs.io/en/latest/)

Besides, two test images have black area where the pixel values are 0. I'd like to know how you deal with it when calculating SAM and UIQ.

[lanha]

Given the fact that you get the published results for RMSE and RSE that suggests that there is some problem in computing SAM and UIQ and not with the result of DSen2.

I am not computing the SAM and UIQ for pixels with values of 0. That means these pixels are not used to compute the average over the whole image.

[Lithsun]

Are those SAM and UIQ functions provide by sewar same as yours? RMSE and SRE are calculated per band and then averaged. Is this the same way to calculate SAM and UIQ per band?

[lanha]

It appears to me that sewar is reporting the result of SAM in radians, whereas in the paper we report the SAM in degrees. For SAM you cannot compute the value per band, it has to be computed per pixel and then averaged over the whole image for each pixel.

def SAM(y_true, y_pred):
    num = np.sum(np.multiply(y_pred, y_true), axis=2)
    denom1 = np.sqrt(np.sum(np.square(y_true), axis=2))
    denom2 = np.sqrt(np.sum(np.square(y_pred), axis=2))
    div = np.divide(num, np.multiply(denom1, denom2))
    tmp = np.clip(div, -1, 1)
    sam = np.arccos(tmp)
    ind = ~np.isnan(div)
    sam = np.mean(sam[ind])*180/np.pi
    print('SAM:    {:.5f}'.format(sam))
    return sam

The UIQ measure is harder to share the code. The value of UIQ is actually computed per band. You can have a look here: https://bitbucket.org/kuraiev/pymetrikz/src/default/

[Lithsun]

Thanks for your help! After recalculating, SAM and UIQ values are correct.