fooof-tools / fooof

Parameterizing neural power spectra into periodic & aperiodic components.
https://fooof-tools.github.io/
Apache License 2.0
363 stars 98 forks source link

Developing a system for benchmarking #304

Open TomDonoghue opened 1 year ago

TomDonoghue commented 1 year ago

There has been some discussion of developing more of a consistent setup / approach for profiling the module / models, and using this as we go forward.

Recent optimizations (https://github.com/fooof-tools/fooof/pull/299) spurred this discussion including the following comment (to keep in mind) about further work on benchmarking: https://github.com/fooof-tools/fooof/pull/299#issuecomment-1694053916

For some basic record keeping, in terms of model fitting, we currently spend basically all of our time in curve_fit, with there not being much obvious opportunity to speed things up beyond what was already done in #299. See this comment (https://github.com/fooof-tools/fooof/pull/299#issuecomment-1687030022) for some comments on potentially exploring different fitting approaches.

In terms of some basic / benchmarking sims / tests, there are some starting points:

I think the general idea would be to develop anything like this with or after the 2.0 release - so for now we can use this issue for any further discussion, and keep this in mind as we move towards 2.0 and beyond.