Reversed results for soundscape saturation

[ghost]

My question

Hello @atruskie, after using AP to calculate acoustic indices, I calculated the soundscape saturation index in RStudio using the PMN index. For some reason, when I plot the soundscape saturation, the pattern is reversed. I checked my code to make sure that I wasn't using -1 or subtracting the saturation values before plotting and I still can't understand why the plot is flipped. Zuzana also checked my code, which is based on the code she used to plot the soundscape saturation for Indonesia and Papua New Guinea and she didn't see anything unusual. I was wondering if the problem is possibly coming from the acoustic indices calculation for my data. Maybe the values in the PMN.csv files are reversed?

I used the following tutorial last June to calculate the acoustic indices: https://ap.qut.ecoacoustics.info/tutorials/01-usingap/practical?tabs=windows

Diagnostics

• I have attached a log:

• The Operating system I'm using:

• The version of AnalysisPrograms.exe:

I was using the 20.6.0.208 version of AP.exe.

• The data used can be found:

At this link is the calculated PMN acoustic indice CSV files that I used: https://drive.google.com/drive/folders/1bGAOXyAF6QrQ9eOctwcnQeAc2groBAJm?usp=sharing

The code that I used in R is at: https://drive.google.com/drive/folders/1CFFuSb0p0KcyD0O8Oq1kb1jnxphBPu5p?usp=sharing

Here is an image that shows the graph for Gabon and Papua New Guinea:

The pattern for Gabon should be similar to the one for Papua New Guinea but it's reversed.

• The command I used:

<!-- 👆 -->

- **The config file I used:**  <!-- Drag and drop the config here -->
<!-- 👇 Drag and drop the log into the empty line below -->

<!-- 👆 -->

[atruskie]

Looking at your R code, it's likely the only line that matters is line 62, where you do your normalization:

import.list$sat1db <- apply(import.list[,2:257],MARGIN=1,function(x) length(x[x>1.55])/2.56)

Why did you pick those values for normalization? I'm not an R expert but it looks like you're trying to calculate the number of frequency bins that have a value over 1.56, and then you convert it to a fraction by dividing by the total number of frequency rows. You use the value 2.56 but the number of frequency bins is 256. Is this an error?

I have a further question: how was the PNG graph derived?

[ghost]

Hi @atruskie,

Thank you for replying back so promptly.

I used 1.55 as the threshold to get a close-to-normal histogram for the data that I am looking at for Gabon. I decided on this number by plotting the histogram multiple times and selecting the number that gave the histogram a fairly normal pattern.

I'm dividing by 2.56 instead of 256 because I want the saturation to be between 0 and 100 (as a percentage) rather than 0 to 1 since it is easier to understand and read.

Regarding your question about the PNG graph, it was generated using a similar code in R. My code is a modified version of the code used by Zuzana.

One update: I ran my code on the data that Zuzana used for Indonesia and my code gives the same pattern as the one she found. My conclusion is that either the saturation pattern in Gabon is the opposite of the pattern in Indonesia and Papua New Guinea, which we think is a bit odd, or values for the indices calculated are off for the site in Gabon.

I was wondering if there were any major changes to the tutorial that I used to calculate the indices using AP.exe since the time Zuzana did her own saturation calculation for the Asian sites. Maybe this could explain why the data is reversed. If not, it is possible that the pattern is simply reversed in Gabon. I will continue to check on my side for other possible explanations.

Thank you so much for your help with this.

[atruskie]

Ok good you isolated the difference between the two datasets.

An aside: I'd suggest avoiding the use of magic numbers in your code. E.g. if you had labelled the 1.55 with a named variable it would have been easier to read. Also compounding operations (like combining 256 / 100 into 2.56) obscures your intention and makes the code harder to read. In the scope of things, it doesn't make the computation significantly faster, or simpler.

The remaining question is did we change anything?

To answer that I need to know which version of AP.exe was used in the original PNG data?

You'll find the version in the log files next to the indices.

[ghost]

Hi @atruskie,

Thank you for your observations. You made some great points regarding using variables rather than magic numbers and compounding operations. I will make sure to avoid doing so in the future to make the code easier to read.

Looking at the log files, the version of AP.exe used for Indonesia was 20.4.0.142. Below is the graph for Indonesia, which has a similar pattern as the graph for PNG:

For PNG, I will need to get the data from Zuzana to know which version of AP.exe she used.

Another point I forgot to mention is that in the initial code from Zuzana, she used the POW indice to calculate the saturation in R for PNG and Indonesia. For Gabon, I'm using the PMN indice, which from my discussion with Zuzana is the same as the POW indice, except that the noise is already subtracted. I think this doesn't matter though since I used the PMN indice to plot the saturation for Indonesia with my code and obtained the same pattern as Zuzana did when she used the POW indice.

[atruskie]

Okay, a couple of points:

• You should consider every version of AP.exe completely incompatible with any other version. We say this because it is way beyond our capability to ensure backwards compatibility. The reality is things don`t change that often and the results are often comparable, but the point is, only compare results for analysis if they were done with the same version of AP
• PMN and POW are different indices. POW was retired because it was averaged incorrectly (averaging log values produces skewed results). PMN and POW should serve the same purpose but you shouldn`t compare results between them.
• From your last post: you prepared a graph on PNG+PMN so it was equally comparable with Gabon+PMN? Using the same version of AP.exe? If yes to both, then great work. If you got the expected result for PNG that leads me to believe the expected result is being calculated correctly for the Gabon. Does that logic hold for you?
  • There is no difference in calculation for different files. The same process is applied. The only thing left to think about after that is things like:
    • is the source audio format different?
    • sample rate?
• If every point above is considered, then it is likely that your data is actually different.
  • Let's ground truth some of the data to see if it is different:
    • Can you show me a false colour spectrogram for the extrema (most minimal and most maximal values) of PMN for Gabon and the extrema for PMN for PNG?
    • The same, but for the PMN only image
    • The same, but a standard spectrogram for a minute?

At this stage I'm thinking the difference you're seeing is genuine. I don't think there have been any major changes to PMN. If the above doesn't solve the issue for you, maybe we should set up a call to work through this in person.

Please forgive any absentminded responses I've given; I've been on holidays until today and was dedicating holiday levels of brainpower :smile:

[towsey]

Good afternoon, I have been following this post and would like to add something: there is minimal difference between POW and PMN. They both calculate the average power (in decibels) remaining after the removal of background noise. POW was retired because it did a simple averaging of decibel values which is technically the wrong way of doing things. PMN (power minus noise) averages the decibel log values correctly. In actuality, the difference between the two averaging methods is small and is unlikely to make any difference to the appearance of your plots. Cheers, Michael

[ghost]

Good afternoon, I have been following this post and would like to add something: there is minimal difference between POW and PMN. They both calculate the average power (in decibels) remaining after the removal of background noise. POW was retired because it did a simple averaging of decibel values which is technically the wrong way of doing things. PMN (power minus noise) averages the decibel log values correctly. In actuality, the difference between the two averaging methods is small and is unlikely to make any difference to the appearance of your plots. Cheers, Michael

Hi @towsey. Thank you for this clarification. It is very much appreciated.

Hi @atruskie,

From the points in your last post, I'm understanding that I cannot compare results obtained from two different versions of AP.exe.

I was looking at Indonesia+PMN vs Gabon+PMN but the calculation of indices wasn't done using the same version of AP.exe. The results for Indonesia and PNG were obtained before I joined Zuzana's lab. I calculated the acoustic indices for Gabon with a later version of AP.exe. Zuzana shared with me the results obtained for Indonesia and I used these results to make sure that the pattern for Indonesia was the same as the one that Zuzana had obtained in the past; I did this to make sure that the reversed pattern wasn't due to an error in my code.

The audio format is the same for all three regions. By sample rate, do you mean the duration of recording? I'm comparing 24-hour recordings in Gabon and Indonesia.

Regarding the false-color spectrograms, can you tell me how the spectrograms for the extrema are labeled among the results because I'm not sure which ones you are referencing?

Below is a false-color spectrogram for the PMN in Indonesia at 6:30 am:

Below is a false-color spectrogram for the PMN in Indonesia for 24 hours:

Below is a false-color spectrogram for the PMN in Gabon:

Ideally, I would show you a false-color spectrogram for a 24-hour recording in Gabon but I only have spectrograms for individual 30-min recordings in Gabon.

At this point, I'm considering re-running the acoustic indices analysis using the same version of AP.exe on the data for each country, just to make sure that I'm comparing results using the same software version. I also noticed that some of the spectrograms for Indonesia are not showing any frequency at some hours of the day (such as the one I shared above with you) but maybe this is normal.

As for the absentminded responses, I wasn't aware! Thank you again so much for your help with troubleshooting!

[atruskie]

From the points in your last post, I'm understanding that I cannot compare results obtained from two different versions of AP.exe.

You can but we don't guarantee they're compatible. AP changes a lot, so scientifically speaking, the only valid experiment is one where other variables are controlled for; AP changing version (and thus implementation) introduces a variable in your experiment that you can't easily control for.

To make dealing with this easier, we ensure that:

• every analysis is tied with a log file and version number (sovereignty)
• we keep old versions of AP available so you can compare using old versions (reproducibility)

The audio format is the same for all three regions. By sample rate, do you mean the duration of recording? I'm comparing 24-hour recordings in Gabon and Indonesia.

Sample rate is the number of times per second a recorder will take a measurement of a sound pressure level. You'll see a number like 16000, 22050, 44100, 48000, etc. all measured in Hertz.

This is relevant because all recordings are converted to 22050 Hz before analysis so we can produce comparable results. If a sample rate conversion happened, then acoustic artefacts can be introduced. However, I don't think that's the case here.

At this point, I'm considering re-running the acoustic indices analysis using the same version of AP.exe on the data for each country, just to make sure that I'm comparing results using the same software version. I also noticed that some of the spectrograms for Indonesia are not showing any frequency at some hours of the day (such as the one I shared above with you) but maybe this is normal.

Maybe just do this for a few select recordings? Just so we can verify that the output is as you expect. The frequency line issue was a bug I believe that was fixed in later versions.

---

So, what I meant by comparing extrema:

• your thesis is that "soundscape saturation seems to be inverted"
• to test this, we need to compare a few samples from each area
• the samples that will show the clearest patterns will be the ones which in your soundscape saturation graph are the most extreme (the peaks and troughs)
• so, pick the extrema in your saturation graphs (1 max, 1 min, from Gabon and PNG)
• work backwards to determine which minute & file contributed to that data points of the extrema
• then we can fill in the following table:

	PNG (1 minute, maxima saturation)	Gabon (1-minute, maxima saturation)	Maxima consistent?	PNG (1-minute, minima saturation)	Gabon (1-minute, minima saturation)	Minima consistent?
PMN	Showns lots of activity?	Showns lots of activity?	?	Showns not much activity?	Showns not much activity?	?
Spectrogram	Can we see/hear lots of activity?	Can we see/hear lots of activity?	?	Can we see/hear not much activity?	Can we see/hear not much activity?	?
Result			?			?

If everything appears consistent, and the indices compared are the same, and the AP.exe version is the same, then we can infer that the index is working properly and your soundscapes are just different.

[ghost]

Hi @atruskie,

Thank you for the clarification about the sample rate and extrema. My apologies for the late reply, I was on the road and didn't have a chance to read your message before leaving.

I will review the data and work on filling the table to see if everything is consistent.

I will post again if I have additional questions.

Thank you

[towsey]

Hi, Comparing your two graphs (Gabon vs PNG), I think you have no choice but to take some one-minute samples through your recordings and listen to the differences. I suspect most of the acoustic energy will be coming from cicadas and crickets etc. How do these vary through the day? In the folder where LDFC spectrograms are produced, you will also find 24hour spectrograms for BGN and PMN. Viewing these will be informative because I believe your saturation index is derived from both of these.. cheers, Michael

[ghost]

Hi @towsey,

Thank you for this idea and I apologize for the delay in my response. I went through your suggestion and there didn't seem to be much difference between Gabon and Indonesia when listening to the soundscape, so I decided to re-calculate the acoustic indices for part of the data in Indonesia. The results are consistent with the pattern found by Zuzana, so at this point, it seems that the patterns are truly reversed.

Thank you again for all the help in this investigation!

[atruskie]

Ok I'm going to close this for now.

If you have any more questions, please reopen and we'll discuss further.