Possible problems in PCA processing when image conversion is applied

[claudioasn]

I ran some PCA tests on some Landsat 8 scenes, both L1 (with and without the application of DOS) and L2 (with and without the application of the conversion constants for reflectance).

Below, I'll post the statistical results and then present some considerations.

PCA with the 7 L1 optical bands (WITHOUT DOS application):

PCA with the 7 L1 optical bands (WITH DOS application):

PCA with the 7 L2 optical bands (WITHOUT application of the conversion constants):

PCA with the 7 L2 optical bands (WITH application of the conversion constants):

In particular, what motivated me to do these tests was coming across a PCA that generated PC1 with more than 99% variance. For orbital remote sensing data, I don't usually see this happen.

As shown in the prints above, when the PCA was generated with the original bands (L1 or L2), the variance of the PC1 was more in line with what I usually see. In addition, the eigenvectors were consistent.

However, when the PCA of the corrected data was generated (either the L1 by DOS, or the L2 with the application of the constants), strange values were seen in the covariance and correlation matrices (with everything giving practically the same thing). Consequently, the eigenvectors are also inconsistent and the eigenvalues show PC1 with more than 99% of the variance.

What could be going on for this to happen? Could the conversion to reflectance, either by DOS or by applying the conversion to reflectance to the L2 data, be causing this situation? The original data at DN level, whether L1 or L2, is more coherent.

[semiautomaticgit]

Hello @claudioasn , please update Remotior Sensus and check again the results.

[claudioasn]

Hi @semiautomaticgit

I updated remotior sensus and scp.

Processed Landsat 8 Level 1 data, applying DOS.

Tried to generate PCA in this new bandset. Here are the results:

Once again, i tried to run PCA for the original L1 bands, just to compare. Here are the results:

These are more consistent data. Also, we can see in +1 diagonal in correlation matrix, and PC1 and PC2 variance are ok, not more than 99%.

I guess there's some issue in image conversion, with DOS method and reflectance conversion with L2 metadata too. Maybe the problem is in somepart of the image conversion, because the original bands are ok.

[semiautomaticgit]

Thank you. I'll look into it.

[claudioasn]

Just to add a new information: i checked PCA generation with a band stack file (all bands in a single file). Here is the report:

So maybe, besides the PCA generation issue with image conversion applied before in the bands, also the stack bands is not working fine, because this stack was generated with original bands, without image conversion.

[semiautomaticgit]

Hello @claudioasn ,

it should be fixed in SCP version 8.3 (updating also Remotior Sensus to version > 0.4). Of course, please reopen the ticket in case you still find any anomalies.