hippke / wotan

Automagically remove trends from time-series data
MIT License
147 stars 21 forks source link

may ground based observations benefit from WOTAN? #32

Open MilevaE opened 5 years ago

MilevaE commented 5 years ago

Hi, in my group we normally apply polinomial detrending taking into account different aspects such as airmass, x-y position, FWHM etc. for our ground based observations. May I use the GP detrending algorithm available in WOTAN instead of our typical polinomials... or GP (or other algorithm available in WOTAN) are only for space missions such as Kepler, TESS etc?

Thanks a lot!

hippke commented 5 years ago

A major difference between ground-based and space observations is cadence and gaps. So the best method can be very different. I assume your group has chosen polynomials because they work reasonably well. Most likely nobody has compared other methods for your intended purpose, e.g. blind transit searches (or something else? pulsations?). Depending on your purpose, the best algorithm can be different. This should be rationalized and not chosen randomly.

GPs have performed poorly for blind transit searches. This is because they are, as their name states, Gaussian processes build on Gaussian noise. When you test whether your data is normally (Gaussian) distributed, you will most likely see that it is not. Systematic noise is the root cause of this evil. Thus, GPs will most likely perform poorly. But - don't believe my instincts! Try it!

If you have data clustered in time (e.g., 20 observations on one night, then a gap of 3 days, etc.), you should use wotan with an appropriate window_length (to preserve your science goal) and break_tolerance so that observations of each night are treated separately as sub-lightcurves.

From my experiments with ASAS and similar ground-based data, I recommend a time-windowed robust slider. Something like this might be a good starting point:

flatten_lc, trend_lc = flatten(
    time,
    flux,
    edge_cutoff=0.0,
    break_tolerance=0.01,  # Split each sub light curve with larger separation
    window_length=0.2,  # Check the plots to make sure this is what you want!
    return_trend=True,
    method='biweight')

Feel free to send me a sample light curve and explain your science goal. Then I can give you a better statement of what I think works well and what won't.