Caoxuheng / HIFtool

A toolbox for HSI-MSI fusion/pan-sharpening, including MoGDCN, Fusformer, PSRT, MSST, DCTransformer, iDaFormer, HySure, HyMS, DBSR, UDALN,uHNTC, and pretrained weights
MIT License
16 stars 2 forks source link

Calculation of “SRF matrix” #1

Closed WCWCWCQAQAQA closed 2 months ago

WCWCWCQAQAQA commented 4 months ago

Dear author, I would like to ask you about the weight parameter "SRF" in spectral degradation. I read the SRF matrix under the R key you provided in 'Dataloader_tool/srflib/chikusei_128_4.mat' and you also provided the Sentinel2A_191_13.mat file. In practice, how should we calculate the "SRF matrix" required to degrade to certain bands under the Sentinel2 sensor? I really hope to get your advice! Looking forward to your reply!

Caoxuheng commented 4 months ago

Hi, WCWCWCQAQAQA I believe the process can be mainly divided into three steps:

1. Determining the Overlapping Spectral Bands:

(1). Firstly, it is necessary to obtain the spectral band information of the Sentinel2A sensor and the spectral coverage of the hyperspectral image.
(2). By comparing their spectral ranges, we can identify the bands of Sentinel2A that overlap or cover the bands of the hyperspectral image. These overlapping bands will form the basis for constructing the SRF matrix, while the non-overlapping parts can be filled with zeros.

2.Interpolation to Align Spectral Bands:

(1). Since the spectral resolutions of Sentinel2A and the hyperspectral image may differ, interpolation is required to align the spectral resolution of Sentinel2A with that of the hyperspectral image. (2). Methods such as linear interpolation or spline interpolation can be used, depending on the nature of your data. The goal is to ensure that the SRF of Sentinel2A is aligned with the band of the hyperspectral image. Through interpolation, a new SRF matrix can be obtained, with the number of columns matching the number of bands in the hyperspectral image and the number of rows corresponding to the target bands of Sentinel2A.

3. Normalization to Preserve Response Distribution:

(1). Normalization aims to ensure that the sum of SRF values for each low-dimensional band (i.e., Sentinel2A band) is equal to 1, thus preserving energy conservation. (2). By normalizing the SRF matrix, we can ensure that the total intensity of the hyperspectral image remains consistent before and after degradation using the SRF matrix.

Also, I would like to recommend a literature that provides a comprehensive understanding of the characteristics of SRF: the paper "A convex formulation for hyperspectral image super-resolution via subspace-based regularization" by TGRS in 2015. Section IV of this paper offers a detailed description of SRF, which may be helpful to you.

If you have any further questions, please feel free to ask.

WCWCWCQAQAQA commented 4 months ago

Thank you very much for your careful reply! However, in response to your reply, I would like to ask you a question about how to calculate the specific value of the SRF matrix. When I read your article "Unsupervised Hybrid Network of Transformer and CNN for Blind Hyperspectral and Multispectral Image Fusion", Reading that you are simulating spectral degradation based on the IKONOS-like reflection spectral response filter in your other paper "Hyperspectral image super-resolution via spectral matching and correction", Is the specific value of the SRF matrix obtained in this process? Since my organization cannot access the article, could you please share it with me? If you can share, I will be very careful to study and strive to achieve! Looking forward to your reply again!

Caoxuheng commented 4 months ago

For information on the IKONOS-like reflection spectral response filter, you can refer to the articles "Fast fusion of multi-band images based on solving a Sylvester equation" from TIP-2015 and "Fusing Hyperspectral and Multispectral Images via Coupled Sparse Tensor Factorization" from TIP-2018.

Moreover, you mentioned that the article "Hyperspectral image super-resolution via spectral matching and correction" primarily focuses on the fusion of visible light images captured by multiple cameras and does not delve into remote sensing imagery.

Caoxuheng commented 2 months ago

Time Out
If you have any further questions or concerns, please feel free to reopen the issue.