Consistent pattern in poorly fit low frequency portions of the spectrum

[jtmiles]

Hello and many thanks for the awesome toolbox!

Everything is very intuitive and typically works great, but I have noticed a tendency to get fits that leave positive residual power in the low end of the spectrum, typically prior to the knee (roughly from 1 to ~5 Hz in our data). I haven't actually quantified this, but it looks almost invariably like fits with larger positive residuals prior to the knee are accompanied by negative residuals just at/within the knee's bend.

Here's a good example of what I'm talking about:

My intuition of what the desired behavior should be is an aperiodic model that is very close to the lowest few frequencies in the spectrum (up until about 6 Hz in the example image), then sits above the spectrum for a few Hz before hitting the linear part of the slope (just shy of 20 Hz in the example). In other words, the knee seems a bit more bent than it ought to be, while the offset is a bit too negative.

I could write something to refit when I hit this issue but I wanted to see if you all had any advice on parameter tweaks that might solve the problem.

I'm using the stable version of fooof (this repo), with:

fm = FOOOF(peak_width_limits = [3, 20], max_n_peaks = 6, aperiodic_mode = 'knee')
fm.fit(freqs, spect, [1, 100])

Any tips would be much appreciated!

[TomDonoghue]

Hey @jtmiles - sorry for the slow reply here. Long story short is that I would say you are not missing anything here per se, this is a relatively common situation we see when looking at (I'm assuming) intracranial data over a broad frequency range.

The reason you're seeing "too much knee" is not exactly about the knee, it's because the 'knee' model we fit adds an exponent (to the higher frequency range), fits a knee as a bend, and then assumes that the exponent below the knee decays to zero. This mean that given a PSD like you show in the example (which clearly has a knee, but who's exponent before the knee is not quite zero), the best the model can do is fit an approximate knee the best it can, and undercut the lowest frequencies, failing to capture the degree to which this section is not flat. This original formulation (the Lorentzian function) comes from some of the theoretical work (which would predict the flat exponent pre-knee) but in real data, for whatever reason(s) this isn't always the case.

In terms of what to do - with the version you are using, there is not anything to do that really addresses the problem, as it's a limitation of the model being fit. Depending on the goals of the analysis, you might tweak things like the frequency range to (for example) skip the lower frequencies and simply fit the higher range better. More broadly, expanding beyond the limited model options in version 1 is a key motivation for the work going into version 2, which will include a much broadened model selection - including a "double exponent" model, in which the model can fit a non-zero exponent below the knee, which should allow for properly modelling cases such as you see there! This is not quite ready yet - but keep an eye out, as there should be a test-version with the new model fitting available soon!

[jtmiles]

Thanks for the thoughtful reply, @TomDonoghue! That all makes sense. Looking forward to the expanded model selection in the next version. Thanks again for all of the work that's gone into this, it's an immensely valuable tool!