255 dcdft standard error of the frequency semi amplitude

[dbenn]

Your review would be appreciated Max.

[mpyat2]

Hi @dbenn ,

I've tried V965 Cep data (see attachment) V965_Cep-V-HJD.tsv.zip

1) DCDFT with period range (Pmin= 0.0850, Pmax= 0.0851, Resolution= 0.0000001) gave the top period= 0.0850674. I've clicked [Create Model], 1 harmonic and got the following uncertainty: Standard Error of the Frequency: 0.00000159477 Standard Error of the Semi-Amplitude: 0.00151954642

2) Then I reload the data (New star from file...) and made the Fourier model with Analysis->Fourier Model... The 'Uncertainty' pane of the 'Model Information' dialog showed:

Standard Error of the Frequency: NaN Standard Error of the Semi-Amplitude: 0.00151954642

Is it normal?

[mpyat2]

Hi @dbenn , I've compared the uncertainty with one from Peranso. Peranso (using DCFT) gives the main frequency = 11.75537 (essentially the same as VStar) with the uncertainty= 0.000055 (much bigger). There are different methods of the calculations of the uncertainty and they may give very different results.

The method of the uncertainty calculation used by Peranso is described in the Peranso help, I've copied the description here, maybe you'll find it useful:

Peranso determines the minimum period error or period uncertainty of the dominant period P, by calculating a 1-sigma confidence interval on P, using a method described by Schwarzenberg-Czerny (1). This method is a so-called ‘post-mortem analysis’ and measures the width and heights of peaks/valleys in a Period Window.

In his paper, Schwarzenberg-Czerny points out that most of the classical error estimation methods (some of which are present in other period analysis software) are unreliable. That’s the main reason why these methods are not supported in Peranso, despite their simplicity and speed of calculation.

The period uncertainty method of Schwarzenberg-Czerny requires the so called Mean Noise Power Level (MNPL) in the vicinity of P. The 1-sigma confidence interval on P then is equal to the width of the line at the P – MNPL level.

The 1-sigma interval corresponds to the width of the line at level MNPL down from the peak

Finding the MNPL may require some care in practice, as many low-power features appearing in a Period Window, are not due to noise but are window patterns of some periods, and thus should not be taken into account. Peranso automatically calculates an approximated MNPL value to determine the period uncertainty. However, you may decide to estimate the MNPL yourself and to enter its value in the Mean Noise Power Level form. The human eye appears to be a good MNPL estimator: simply look at the Period Window and estimate the mean level of the power spectrum (or equivalent) around P, ignoring all strong lines and their aliases.

(1) Schwarzenberg-Czerny, A., 1991, Mon. Not. R. astr. Soc., 253, 198-206.

[mpyat2]

On the other hand, for this sample data, Peranso's uncertainty estimation of the main frequency looks too conservative. I tried to build phase plots with 1/(Freq - uncertainty) and 1/(Freq + uncertainty) and I saw a big difference; using visual estimation of the best frequency (period) gives a more 'sensible' error than formal estimation. VStar's estimation looks a bit too optimistic (at least 'visually').

[dbenn]

Thank you for all the useful input @mpyat2. The implementation here was driven by the forum discussion referred to in #255. My main take away from this was to focus initially on the metrics suggested by Grant Foster, here and in #256.

I definitely take your points. My current thinking however, is to start with the standard errors (#255) and FWHM (#256) and leave open other measures of period uncertainty in future, and indeed to include as many as possible, much in the same way we include AIC, BIC, RMS for polynomial fit models and others are being considered for that (#258).

Also, as Foster also says (page 154 of Analyzing Light Curves):

In most cases the possible range estimated by the theoretical formula is too liberal while the range estimated by FWHM is too conservative.

So, I'm hoping we'll get a counterbalanced view of period error initially this way. Then in a future release, we add more, whereas right now, we have none.

Does that seem reasonable?

[dbenn]

@mpyat2, the Standard Error of the Frequency NaN error you found is definitely a bug. I'll investigate. Thanks!

[mpyat2]

Yes, it is reasonable. For now, it is a good starting point.

[BradWalter]

I agree supplying both the analytical Standard error of the peak frequency and the FWHM. The standard error almost always underestimates and the FWHM almost always overestimates so you get estimates of lower and upper boundary.

Please note at least for FWHM (not sure about Standard it is important that frequency spacing not be larger than the recommended Frequency grid spacing in eq 7.9 when estimating the FWHM since observations are not evenly spaced in time. The width changes as you increase spacing between frequency samples Particularly above the critical frequency spacing (Δν = 1/(Nτ) if samples are equally spaced in time).

Also It must be clearly understood that the “A” in the standard error formula is what foster calls the semi-amplitude (absolute value of difference between average and a peak) which is what engineers call Peak amplitude. Amplitude, as used by Foster is what engineers call peak-to-peak (positive direction peak – negative direction peak).

Brad

From: dbenn @.> Sent: Thursday, May 5, 2022 9:35 AM To: AAVSO/VStar @.> Cc: Subscribed @.***> Subject: Re: [AAVSO/VStar] 255 dcdft standard error of the frequency semi amplitude (PR #262)

Thank you for all the useful input @mpyat2 https://github.com/mpyat2 The implementation here was driven by the forum discussion referred to in #255 https://github.com/AAVSO/VStar/issues/255 . My main take away from this was to focus initially on the metrics suggested by Grant Foster, here and in #256 https://github.com/AAVSO/VStar/issues/256 .

I definitely take your points. My current thinking however, is to start with the standard errors (#255 https://github.com/AAVSO/VStar/issues/255 ) and FWHM (#256 https://github.com/AAVSO/VStar/issues/256 ) and leave open other measures of period uncertainty in future, and indeed to include as many as possible, much in the same way we include AIC, BIC, RMS for polynomial fit models and others are being considered for that (#258 https://github.com/AAVSO/VStar/issues/258 ).

Also, as Foster also says (page 154 of Analyzing Light Curves):

In most cases the possible range estimated by the theoretical formula is too liberal while the range estimated by FWHM is too conservative.

So, I'm hoping we'll get a counterbalanced view of period error initially this way. Then in a future release, we add more, whereas right now, we have none.

Does that seem reasonable?

— Reply to this email directly, view it on GitHub https://github.com/AAVSO/VStar/pull/262#issuecomment-1118631070 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ACAV73LA3YS6YBTSBB23WJ3VIPMB3ANCNFSM5U4DU4ZQ . You are receiving this because you are subscribed to this thread. https://github.com/notifications/beacon/ACAV73NNMAKWMRSRIXPHD6DVIPMB3A5CNFSM5U4DU4Z2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOIKWPJHQ.gif Message ID: @. @.> >

[dbenn]

Thanks for the additional info @BradWalter. Yes, "A" is the semi-amplitude taken from the DCDFT result in the implementation.

[dbenn]

@mpyat2 The bug you found arises from the fact that semi-amplitude (A) is used in the calculation of standard error of the frequency and is taken from the DCDFT result for the selected period. When a Fourier model is created independent of a DCDFT, semi-amplitude is not available. So, what I suggest is that for Fourier models, standard error of the frequency is not reported. See next commit.