spectralpython / spectral

Python module for hyperspectral image processing
MIT License
574 stars 139 forks source link

How to use SPy to train one model on multi images? #149

Open Jasonwtw opened 1 year ago

Jasonwtw commented 1 year ago

Hi there,

I'm new to hyperspectral image classification and find out SPy is really a useful tool. But I wonder how to use SPy to train one model on multi images, just like what usually do in RGB image classification.

Any information would be greatly appreciated.

Thx.

tboggs commented 1 year ago

A brute force way to do this would be to ravel each image and groundtruth labels so they have shape (1, N, n_bands) and (1, N), respectively. Then, concatenate them into a single flattened image of shape (1, M, <n_bands>) and groundtruth image of shape (1, M).

If you are feeling more ambitious, you could create a MultiImageIterator that wraps multiple spectral.algorithms.ImageIterator objects for an unsupervised classifier or create a MultiImageMaskIterator to wrap a list of spectral.algorithms.ImageMaskIterator objects to create a supervised classifier. It's probably not more than a dozen lines of code.

Jasonwtw commented 1 year ago

Thanks for the response.

I will try out your suggestion for wrapping the ImageIterator objects. But another question is if this method is suitable for image classification algorithms such as CNN, where the image could be analyzed as batches by window sliding.

tboggs commented 1 year ago

The methods I described above are iterating over pixels for training a model that does pixel-level classification. If you want to do classification of images (i.e., considering the spatial context of the spectra), then you need to load entire images (or regions of images) into memory so your convolutional layers can process the 3D data (rows, columns, & bands).

You could simply load all of the images into memory (e.g., using the load method of the SpyFile object) but if you have a large number of images, that may consume too much memory, in which case you might need to load them on-demand.