search

CHIMEFRB / fitburst

An open-source package of utilities for direct modeling of radio dynamic spectra.
https://chimefrb.github.io/fitburst/
MIT License
11 stars 2 forks source link

Use Information criterion or Bayesian evidence instead of chi squared to determine best model #58

Open DanielaBreitman opened 2 years ago

DanielaBreitman commented 2 years ago

When fitting with multiple models where we vary the number of peaks, currently, we use the chi squared to determine the best model with the least parameters. Better practice would be to change this to use Akaike Information Criterion (AIC) or some other IC OR, if not too difficult to extract, the Bayesian evidence.

emmanuelfonseca commented 2 years ago

@DanielaBreitman I added an initial stats.py file in fitburst/analysis, which only contains a function to compute an F-test statistic. however, F-tests are certainly not the only thing and I definitely agree that AIC and equivalent tests should be available and used. how about we add one or a few additional functions, like AIC and and Bayes-factor calculations, into this file?

DanielaBreitman commented 2 years ago

This is perfect! This is the way to proceed. Thanks @emmanuelfonseca !