simedroniraluca
commented
9 months ago Parameters Analysis
The background noise detection implemented in oceanstream is based on the remove_noise function from the echopype library, which in turn is based on the techniques described in the article by De Robertis & Higginbottom (2007), titled 'A post-processing technique to estimate the signal-to-noise ratio and remove echosounder background noise,' published in the ICES Journal of Marine Sciences (64(6): 1282–1291).
The parameters for oceanstream's apply_remove_background_noise function, designed to address background noise issues, are as follows:
ds_Sv: xr.Dataset- Dataset containing Sv and echo_range [m].
ping_num: int, default=40- Number of pings to obtain noise estimates.
range_sample_num: int, optional- Number of samples along the
range_sampledimension to obtain noise estimates. If not provided, it's computed based on a 10-meter vertical bin for each channel, and the minimum value is used.
- Number of samples along the
noise_max: float, default=-125- The upper limit for background noise expected under the operating conditions (default=-125 dB).
SNR_threshold: float, default=3 [dB]- Acceptable signal-to-noise ratio.
The default values for the parameters in oceanstream's apply_remove_background_noise were chosen based on the default values tested in the article for calibrated Simrad EK60 echosounders operating at 18, 38, 120, and 200 kHz:
- Horizontal cell size of 40 pings
- Vertical cell size corresponding to a 10-m vertical bin
- An acceptable signal-to-noise ratio defaulting to 3 dB
- Noise max = -125 dB
Other important considerations highlighted in the article that should be taken into account when adjusting the input parameters for the apply_remove_background_noise include:
- It is advisable that the
SNR_thresholdbe greater than 0. noise_maxrepresents an upper limit for background noise expected under the operating conditions. It must be determined empirically and will depend on the echosounder. In the examples presented in the article,noise_maxwas set at -125 dB at all frequencies, and this value was never exceeded.- 'The magnitude and the variability of the noise estimates rapidly level off as increasing numbers of pings are averaged, indicating that they are relatively insensitive to grid size when more than 20 pings are averaged, and 10-m deep cells are used. As a compromise between the stability of the noise estimate and the ability to compensate for rapidly changing noise levels, cell-size parameters of N = 40 pings in the horizontal and M corresponding to a 10-m vertical bin in the vertical were used in the analyses.'
When applying background noise removal with the following parameters:
"remove_background_noise": { "ping_num": 40, "range_sample_num": 10, "noise_max": -125, "SNR_threshold": 3, }We lose around 60-65% of data as noise. This should be investigated to make sure we arent eliminating to much useful data.