tlecomte / friture

Real-time audio visualizations (spectrum, spectrogram, etc.)
http://friture.org
GNU General Public License v3.0
909 stars 159 forks source link

high frequency artifacts in spectrograms with reduced max_freq #132

Closed egitto closed 4 years ago

egitto commented 4 years ago

image

Steps to reproduce:

  1. Open Friture
  2. Reduce the 2D spectrogram's max frequency to 1000 Hz
  3. With a descending tone, speak a vowel with a lot of brightness, like an æ ("aaaa") sound
  4. Observe that (some of?) your harmonics mirror, and go up and down all over the place, instead of only floating above and parallel to your f0 like harmonics should

I understand that this is due to the decimator using the "simplest way"

    def decimate(self, samples):
        # first we remove as much points as possible
        if self.decimation > 1:
            samples.shape = len(samples) // self.decimation, self.decimation
            # the full way
            # samples = samples.mean(axis=1)
            # the simplest way
            samples = samples[:, 0]
        return samples

I propose using the full way rather than the simplest way, as that should mostly eliminate this problem (as the higher-than-threshold frequencies should approximately cancel themselves out), while maintaining the presumable performance benefit of giving a smaller set of samples to the FFT.

At the moment, the decimator seems to be optimizing performance when the window is cropped, but at the cost that the spectrogram has ghosts if there are significant harmonics above some threshold frequency. I expect that in most use cases of the app, there will be significant harmonics; human vocalizations and musical instruments are full of them; so probably this is also the case for times when one would want to crop the window?

I have created a PR for this. Some additional details including costs and benefits are there. https://github.com/tlecomte/friture/pull/133

tlecomte commented 4 years ago

Thanks @egitto for this very clear report ! Obviously we have a sampling problem... As you found, decimating without pre-filtering is a perfect recipe for artifacts... I guess I wrote that part when I was not clear enough on sampling theory!

I'll try to look at whether there is a fast-enough and correct-enough way to do that. Using an average as you did in #133 is a step in the proper direction, but we might be able to do better (by using a filter to do the downsampling instead of an average). I don't have a lot of spare time these days, so I can't promise any ETA unfortunately.

tlecomte commented 4 years ago

I've removed the decimation completely in #137 Thanks again for the report. Please reopen the issue if it does not work ! Thanks !