Spectral Signature Plot does not correctly calculate the ROI values in each band

[claudioasn]

I'm using SCP version 8.1.3, with remotior-sensus updated. When I generate the ROIs and calculate their spectral signature, when I display it on the plot it looks as if the value is the same in all the bands. Until version 7 this worked normally, but now it's giving me this problem. The scatterplot presents the same problem.

[semiautomaticgit]

Hello, please update Remotior Sensus. If you still get the same issue, please copy the log file following these steps https://semiautomaticclassificationmanual.readthedocs.io/en/latest/faq.html#how-can-i-report-an-error

[claudioasn]

Hello, I already have the SCP and remotior-sensus updated, I had them before.

I tried to do the process, but QGIS doesn't exactly display an error message. I tried to do the whole process and save the .txt file, but it comes up blank.

What I did was open the landsat 8 bands, generate the DOS process, make a stack with the 7 optical bands, define the stack bands as bandset and create the ROIs for spectral plot.

It seems that SCP (or remotior-sensus) only captures the pixel value of one of the bands, and not all of the bands in the stack, or even all of the bandset, and the plot is not displayed correctly.

NOTE: I don't know if I need to open another topic, but I noticed another problem: with a bandset defined from a stack of bands (I tested with several scenes) there was a problem when making a PCA. The covariance matrix has the same value in all cells, the correlation matrix is empty, and the eigenvectors and eigenvalues also have problems. Before version 8 of SCP, which has the remotior-sensus dependency, I did exactly the same procedure and it worked normally.

Here is the PCA .txt file.

PCA_N.csv

[semiautomaticgit]

Hello @claudioasn ,

What I did was open the landsat 8 bands, generate the DOS process, make a stack with the 7 optical bands, define the stack bands as bandset and create the ROIs for spectral plot.

stacking bands is not required as you can directly define the bandset using single bands. Could you please try without stacking bands?

[claudioasn]

Hello @semiautomaticgit , when I used the separate bands the spectral plot worked for the rois. However, when I generate a stack of bands and load it as a bandset, this problem occurs. It's as if it captured the value of only one of the bands and not all seven, showing the same value for all of them.

Could the problem be in the construction of the stack and/or when it is loaded into the bandset? I've tried saving a log file but no error is generated. It is saved empty.

Can you try to simulate this situation and see what happens for yourself? Make a bands stack, load it into bandset1, set the wavelenght, generate any roi and view the spectral plot?

Here's a screenshot of how the stack I load is configured.

EDIT: When I load the separate bands, scp automatically differentiates the bands correctly in the center wavelenght column, either by band order or by selecting the landsat option. However, when I load the stack as a bandset, the center wavelenght column shows a single value for all the bands, and only changes when I change the selection option in the band quick settings. Is this a possible problem? The SCP could somehow be thinking that all the bands in the stack are just one, no matter how much I change the values in the band quick settings.

As an example, below is an image of when I load the bands separately. Note that the values are different.

And now, when I load the stack. See that the wavelenght value is the same for all the bands, as if somehow, in some metadata, they are all understood by the SCP to be the same wavelength. However much I change the values in the band quick settings, the spectral plot only loads one wavelenght.

Taking advantage of the topic, because I've also talked about problems generating PCA before. I believe that this problem in the stack is also what is causing the problem in generating the PCA.

I tried to generate the PCA with the separate bands after generating the DOS, below are the tables generated. See how the correlation and covariance matrices look, all the same values, with the first pc having a variance of over 99%

So out of curiosity I tried to generate the PCA with the L8 level 1 bands (before the DOS correction), and apparently it worked correctly, as shown in the image below. So the question remains as to which step the problem is occurring: if already in the DOS generation or if in the generation of the Stack. I have another topic open that says that when I try to make the DOS correction and load the metadata file, SCP gives an error (shown in the other topic). Could this be the start of what's causing all these problems?

[semiautomaticgit]

Hello @claudioasn ,

thank you for reporting this.

Could the problem be in the construction of the stack and/or when it is loaded into the bandset? I've tried saving a log file but no error is generated. It is saved empty.

It is quite strange that the log file is empty, there could be some installation issues.

Can you try to simulate this situation and see what happens for yourself? Make a bands stack, load it into bandset1, set the wavelenght, generate any roi and view the spectral plot?

The issue is related to Remotior Sensus library, and the way it manages multiband raster. For the moment, I recommend using single band rasters when creating band sets.

So out of curiosity I tried to generate the PCA with the L8 level 1 bands (before the DOS correction), and apparently it worked correctly, as shown in the image below. So the question remains as to which step the problem is occurring: if already in the DOS generation or if in the generation of the Stack. I have another topic open that says that when I try to make the DOS correction and load the metadata file, SCP gives an error (shown in the other topic). Could this be the start of what's causing all these problems?

I'll try to check if there are issues in the DOS process. However, if you are using Landsat collection 2 I recommend using level 2 images which are already atmospherically corrected with a method that is more accurate than DOS.

[claudioasn]

Hello, @semiautomaticgit, I've tried again in various ways. Making a bandset only with the separate bands after performing DOS, making a stack bands... in short, there is always some problem, and I have tested it on more than one computer. Both problems with the spectral plot after dos and stack bands, as well as generating pca

As for the level 2 collection, I'm going to try using it, but as I teach remote sensing I like to show the procedure for correcting level 1 data or data analogous to it as well.

I checked here that my python version in qgis is 3.9, and not 3.10 (recommended by remotior-sensus). Could this be causing these problems? How can I upgrade it via the osgeo4w shell?

[semiautomaticgit]

Hello, @semiautomaticgit, I've tried again in various ways. Making a bandset only with the separate bands after performing DOS, making a stack bands... in short, there is always some problem, and I have tested it on more than one computer. Both problems with the spectral plot after dos and stack bands, as well as generating pca

Hello @claudioasn , thank you for the feedback, I'll look into it.

As for the level 2 collection, I'm going to try using it, but as I teach remote sensing I like to show the procedure for correcting level 1 data or data analogous to it as well.

I understand.

I checked here that my python version in qgis is 3.9, and not 3.10 (recommended by remotior-sensus). Could this be causing these problems? How can I upgrade it via the osgeo4w shell?

It could be related to Python version, but unfortunately I don't know how to upgrade it via osgeo4w.

[claudioasn]

Hi @semiautomaticgit

I tested again this issue in new remotior sensus and scp version. The problem continues with the stack bands. With separated band files in a single bandset, it works, but with a stack bands file it does not.

[semiautomaticgit]

Hi @claudioasn , it should be fixed now. Please update both SCP and Remotior Sensus. Thank you for your feedback.

[claudioasn]

Hi @semiautomaticgit

I confirm that now the spectral plot is working with de band stack, even with the DOS files. However, the PCA still has problems with the DOS bands. I works with the original L1 or L2 bands, without image conversion. I tested with bandsets composed by separated bands and a bandstack, but this issue i think is related to another topic i opened already.