search

spectralpython / spectral

Python module for hyperspectral image processing
MIT License
574 stars 139 forks source link

Spectral angles calculation with invalid values #132

Closed johngallino closed 2 years ago

johngallino commented 2 years ago

I am trying to calculate spectral angles by comparing 18 specific bands of my image to 10 custom spectra classes.

My classes are in an array of shape (10, 18) My subimage of 18 bands is in shape (2905, 256, 18)

when running the spectral_angles calculation and passing the two arrays, I'm getting the following:

W:\Anaconda3\envs\HyperGix\lib\site-packages\spectral\algorithms\algorithms.py:1354: RuntimeWarning: invalid value encountered in sqrt norms = np.sqrt(np.einsum('ijk,ijk->ij', data, data))

sometimes with other files I'll also get this error

W:\Anaconda3\envs\HyperGix\lib\site-packages\spectral\algorithms\algorithms.py:1356: RuntimeWarning: divide by zero encountered in true_divide dots = np.clip(dots / norms[:, :, np.newaxis], -1, 1)

I'll admit I'm not equipped to fully understand the math going on here. I know that my images (from the Hyperion satellite sensor) contain some bands with 0 and negative values, but I feel like there should be a contingency plan for that for the spectral angle calculation? I'm not sure what else could be causing the errors, but it's causing my spectral angle matrix to mostly consist of NaN. Is it something wrong with the way I'm manipulating my data? Note that I am running Python 3.8.

tboggs commented 2 years ago

My guess is that you have NaN values in your data. You can test that with

>>> np.any(np.isnan(your_image_data))

If that expression evaluates to True, then you have NaN values in the data, which you will need to remove (e.g., if it is a bad band, you could remove it from the image & classes or replace it with the average of adjacent bands).

I don't believe having zeros or negative values in the data is a problem, though having all zeros in a pixel would be a problem because the associated vector can't be normalized.

johngallino commented 2 years ago

I tried your expression on the original image data, on the subimage of 18 bands, and on my classes matrix and all evaluated to False. There's no NaN data anywhere in the inputs.

tboggs commented 2 years ago

Then I don't know what the specific issue is. I recommend you try manually computing the norms of your data and classes to see where zeros or NaNs might appear there. You might also try converting your data/classes to float64, in case there is an overflow issue.

johngallino commented 2 years ago

It seems the overflow was indeed the issue. The read_bands() method creates an npArray of int16. Converting to float64 allowed the calculations to proceed as intended.