Question of the matched filter equation

[zxdawn]

Hi, I have a question about the matched filter applied in your algorithm.

I see you mentioned these two equations:

https://github.com/UCSB-VRL/MethaneMapper-Spectral-Absorption-aware-Hyperspectral-Transformer-for-Methane-Detection/blob/7ae04159e076ca7b5ba4f60c330161bff924b934/train_dataset_generator/match_filter.py#L150-L152

They are like these:

$$alpha(x)=\frac{(\mu_t-\mu_b)^T \Sigma^{-1} (x-\mu_b)}{(\mu_t-\mu_b)^T \Sigma^{-1} (\mu_t-\mu_b)} (1)$$

$$alpha(x) = \hat{a} = \frac{(x-\mu_b)^T \Sigma^{-1} (\mu_t-\mu_b)} {(\mu_t-\mu_b)^T \Sigma^{-1} \cdot (\mu_t-\mu_b)} (2)$$

Eq. 1 is as same as the spectral library. The numerator of Eq. 2 is different from Eq. 1, will they have the same results?

According to the equation in your paper, the alpha is defined as $$alpha(x) = \frac{(x-\mu_b)^T \Sigma^{-1} t}{\sqrt{t^T \Sigma^{-1} t}} (3)$$ Then, why did you remove $\mu_b$ ($\mu_t-\mu_b$) instead of using $t$ directly? If I understand correctly, $t$ is the $\mu_t$ in your code.

[satish1901]

Hello Thanks for pointing out, Equation 3 is the correct equation. We used the spectral library implementation of matched filter for the work.

[zxdawn]

Thanks a lot for your quick reply @satish1901 . Actually, I see these differences in other papers too:

Thompson (2015)

Wang (2023)

Could you point out 1) why the first paper uses sqrt while the other one doesn't 2) why the first paper uses $t-\mu$ while the second paper uses $t$?

I suppose this is the same case for your comments in the code. If you could explain it a bit, I would appreciate it a lot ;)

[satish1901]

Hello

The main matched filter is basically the Numerator, which is same in all the equations. The denominator is only for normalization, that's is why you are seeing slight variations among all the papers for matched filter implementation.

The square root is often dropped for saving on to the mathematical computations. Each equation of matched fitler will give you similar output with slightly variation in the magnitude of values. I hope this clears some confusion.

Thanks :)

[zxdawn]

Thanks a lot! Now, I can understand the concept with your explanation and this paper.

Another thing that confuses me is the normalization:

https://github.com/UCSB-VRL/MethaneMapper-Spectral-Absorption-aware-Hyperspectral-Transformer-for-Methane-Detection/blob/7ae04159e076ca7b5ba4f60c330161bff924b934/train_dataset_generator/train_data_generator.py#L201-L202

Why did you normalize the data for the matched filter? Is that related to the target you defined?

t represents the change in radiance units of the background caused by adding a unit mixing ratio length of CH4 absorption

I see someone else multiply the target (CH4 transmittance) by the mu value.

BTW, how did you calculate the target file? Because I'm trying to apply a similar method to different hyperspectral imagers, I need to generate the gas_signature file by myself.

[satish1901]

Data normalisation is necessary to reduce the intensity variation. This basically put our whole data in same range and avoid surprises

target file is the prior to every gas you are looking for. This is unique to each has molecule. It represent the transmittance of CH4 gas in our case. you can compute this for different gases using HITTRAN API

[zxdawn]

Em ... If the data is normalized, then how did you get the correct CH4 enhancement with units like ppb or ppm? I didn't see the step of converting back from matched filter results, which are based on normalized values.

And thanks for sharing the transmittance source. I will try it.

[satish1901]

We are not converting the matched filter output to the exact values of CH4 enhancement. There are many radiative transfer models to convert those values. In our case, we pass the raw matched filter values to out feature extractor. We just need the information about the relative concentration information, that is provided by the matched filter

[zxdawn]

Ha, got it. Close this issue now. Thanks a lot for all of your responses!

[satish1901]

I hope it was helpful! good luck with your research