CLEANest problems/questions

[dbenn]

Brad Walter writes in email (Nov 2020):

I have a bunch of questions about how CLEANest works. ... The most basic question is adding frequencies that don’t appear in the TopHits List. Seen the attached screenshot where I am trying to add two periods that are not in the TopHits list. CLEANest will let me add one, either one of the two, but when I try to manually add two periods in a single CLEANest operation, CLEANest seems to do nothing. CLEANest takes a minute or two to complete with the periods selected from TopHits and one manually added variable period. With the same TopHits selected and two manually added variable periods, I waited over half an hour and nothing seems to happen. I tried several times and the same lack of response resulted each time. VStar does not crash and I can do more DCDFTs and CLEANest operations as long as I don’t add two stars manually to the variable Period list. I am adding them by typing a period in the box and clicking on add repetitively.

I do not understand the power and amplitude values that show up for CLEANest results in the TopHits list. Grant states in Analyzing Light Curves (pp 254, 255) that CLEANest does a fine resolution scan around the Periods entered (usually by selecting multiple TopHits) and displays thin lines having the amplitudes of the frequency fits of the simultaneously fit signals superimposed on a graph of the residual amplitude spectrum after the frequencies are removed.

That is not what VStar does. VStar seems to simply display Thin vertical lines that all span the range axis of the power plot and their height has no relationship to the amplitude of the various frequencies. The CLEANest amplitude plot appears to be the same as the amplitude plot immediately before performing CLEANest. I don’t understand the meaning of the (identical) power and amplitude values in the TopHits table for all the CLEANest frequencies that were fit. The Power value assigned to all the frequencies is much larger than any of the powers of the individual frequencies before CLEANest is applied. Is it the sum of the refined Power values of the individual frequencies? The amplitude displayed for all the cleanest frequencies is always much smaller than the amplitudes of individual frequencies before applying CLEANest. I have no idea what the semi-amplitude value assigned by CLEANest represents. It certainly isn’t the sum of the refined semi-amplitude values for the selected frequencies. ... Some additional info. If I enter one of the two final period selections that cannot be added from TopHits as a fixed period and the other as a variable period CLEANEST works. It also works if I add both of the final two periods in the fixed box. It still appears not to start the CLEANest process if I try to enter more than one period as variable.

Further I tried using slightly different values for the one period entered in the variable period input box in different CLEANEST attempts (no models created and CLEANests run on the original data in all cases). For example I input 856.52 as the Variable Period input for one try and 856.428 for another. The resulting period output from CLEANest remained the same as the value I input in all cases. It was not refined. I used these two values because when I performed the DCDFT on the previous residual to identify this period at approx. 856 sec the TopHit value before I ran CLEANest was 856.428 before doing CLEANest and 856.52 result from the single TopHit frequency However, there was no TopHit period in the DCDFT of the original data within 1 Second of the TopHit on the residuals pre-whitened with all of the previously discovered frequencies. My methodology is to discover a new top hit and then go back and do a CLEANest including that new frequency plus all of the previously discovered frequencies (13 preceding fundamentals in this case + the 2nd harmonic of the most powerful) and use the CLEANest results + significant harmonics, if any in the model used to generate a new set of residuals on which I run a DCDFT and single frequency CLEANest to see If there is an additional significant frequency to CLEANest with the preceding ones and roll into a next model generation, repeating the progressive forward selection processes.

One would expect that when I tried the pre and post single frequency CLEANest periods in the Variable period box one or both of the CLEANest outputs would be different from the input if in fact CLEANest had the ability to vary the period to obtain the best fit. The periods corresponding both the input and output of the single frequency CLEANest can’t be the right. The output of the multi period CLEANest should differ from at least one of them and potentially from both of them due to the interaction with previously discovered frequencies. My results lead me to believe that the Variable Period choice is treating the input period as a fixed period that it cannot vary based on goodness of fit I hope I am explaining in an intelligible way.

Attached are the inputs and outputs of the two CLEANests where the only difference between inputs is the small change in value of the variable period input. That Small change carried through to the output without change in period although a couple output periods for frequencies corresponding inputs from TopHits (which all have higher power but are regarded as variable) do change very slightly as a result of the change in the 856.xxx second value entered into the variable period input box.

[dbenn]

Brad gave an example of a dataset to which this was applied, but it was large and had many periods. It would be best to try to replicate the problems he is seeing (in particular: handling of variable and locked periods) with a smaller, simpler dataset.

[dbenn]

In email on March 4th 2021, Brad said:

Whenever you add a new frequency in the iterative process you should do another DCDFT from the beginning and CLEANest all the frequencies discovered including the new add. The frequencies interact and that is the way you get the most accurate frequency determination Do a DCDFT Select the top hit and CLEANest it Create a model and subtract it DCDFT the residuals Identify the top hit Go back to the original data DCDFT Select the two frequencies you have identified (or the closest to them in the original data) Run CLEANest using all identified frequencies Create a model using CLEANest results Run DCDFT on the residuals Return to original data repeat the process adding the newly identified frequency each time.

If you have a very complicated periodicity as you can have in some pulsating white dwarfs, DSCT and pulsating sdA/SdB stars you can end up identifying some frequencies that don’t show up in the DCDFT of the original data or their harmonics. For example, they may be cross harmonics of different fundamental frequencies. Those you have to add these manually to CLEANest and they should be added so that the frequency can be modified because of interactions among various frequencies.

[dbenn]

@mpyat2, @clkotnik: some interesting issues in this one re: CLEANest.

[mpyat2]

Hi @dbenn, I'm not an expert on the topic however I did some formal tests. A description of the iterative process in the last message is similar to CLEANest/SLICK procedure implemented in Peranso. See http://www.physics.emory.edu/astronomy/observatory/staff/Dale/hd%20teaching/444-544%20Advanced%20Lab/PeransoUserManual%5B1%5D.pdf, page 59 "Tutorial 3 : Finding Multiple Periods using CLEANEST"

I've reproduced the iterative process described in the Peranso manual with VStar using UW Her data (see the attachment): 1) DCDFT with Frequency Range: Low Freq = 0.0007, High Freq = 0.0150, resolution = 0.00001. 2) Select top period 107.53 d, do CLEANest. 3) The period after CLEANest = 107.6 d (the same as in Peranso) 4) Create a model with the period 107.6 d, save residuals to a file 5) Load residuals 6) DCDFT on residuals with the same parameters as in (1) etc.

I've obtained almost the same results as in Peranso (three frequencies extracted iteratively).

(by the way, VStar does DCDFT much faster than Peranso)

I could not reproduce a problem described in the first message. Probably more checks needed.

UW_Her.zip

[BradWalter]

I don’t know if you included the following step. Whenever you isolate a new frequency by doing a DCDFT on Residuals you go back to the original data and do a model fit with all frequencies discovered to that point, including the new one, to get the next round of residuals on which to do a DCDFT. Therefore the residuals from which you isolate a new frequency are the residuals from a model fit using all the frequencies discovered to date. That is important to include the interactions between the discovered individual frequencies that may cause them to shift in frequency or modify amplitude/power levels. This is the forward selection process described in section 5.6 of Grant’s book and is also a step in the process used by the excellent periodic analysis program Period04: https://www.univie.ac.at/tops/Period04/

Brad Walter

From: mpyat2 @.> Sent: Sunday, March 21, 2021 3:01 PM To: AAVSO/VStar @.> Cc: Subscribed @.***> Subject: Re: [AAVSO/VStar] CLEANest problems/questions (#103)

Hi David, I'm not an expert on the topic however I did some formal tests. A description of the iterative process in the last message is similar to CLEANest/SLICK procedure implemented in Peranso. See http://www.physics.emory.edu/astronomy/observatory/staff/Dale/hd%20teaching/444-544%20Advanced%20Lab/PeransoUserManual%5B1%5D.pdf, page 59 "Tutorial 3 : Finding Multiple Periods using CLEANEST"

I've reproduced the iterative process described in the Peranso manual with VStar using UW Her data (see the attachment):

1. DCDFT with Frequency Range: Low Freq = 0.0007, High Freq = 0.0150, resolution = 0.00001.
2. Select top period 107.53 d, do CLEANest.
3. The period after CLEANest = 107.6 d (the same as in Peranso)
4. Create a model with the period 107.6 d, save residuals to a file
5. Load residuals
6. DCDFT on residuals with the same parameters as in (1) etc.

I've obtained almost the same results as in Peranso (three frequencies extracted iteratively).

(by the way, VStar does DCDFT much faster than Peranso)

I could not reproduce a problem described in the first message. Probably more checks needed.

UW_Her.zip https://github.com/AAVSO/VStar/files/6178192/UW_Her.zip

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/AAVSO/VStar/issues/103#issuecomment-803650379 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ACAV73M7S3USJZWBJIQF6FDTEZF6PANCNFSM4VO6JJDA . https://github.com/notifications/beacon/ACAV73L7J2KKGWXJV5DQ3JDTEZF6PA5CNFSM4VO6JJDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOF7TLWSY.gif

[mpyat2]

Currently, VStar provides an isolated CLEANest procedure only, without extra functionality like Peranso's "CLEANest Workbench" or other similar tools. So, the user is forced to take manual steps. It's a limitation, of course. Probably, @dbenn could shed more light on this...

[dbenn]

Thanks for the UW Her example @mpyat2. Good to know re: Peranso and DCDFT!

I understand your point I think @BradWalter. Let me think about this.

One of the things we also don't implement is SLICK which the TS Fortran code does from which DCDFT, CLEANest etc was ported to Java.

[BradWalter]

I can’t comment re Peranso and comparisons to it. I have never used it.

From: mpyat2 @.> Sent: Monday, March 22, 2021 3:49 AM To: AAVSO/VStar @.> Cc: Bradley Walter @.>; Comment @.> Subject: Re: [AAVSO/VStar] CLEANest problems/questions (#103)

Currently, VStar provides an isolated CLEANest procedure only, without extra functionality like Peranso's "CLEANest Workbench" or other similar tools. So, the user is forced to take manual steps. It's a limitation, of course. Probably, @dbenn https://github.com/dbenn could shed more light on this...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/AAVSO/VStar/issues/103#issuecomment-803880220 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ACAV73OUHEKB7PKBLACCQ33TE376DANCNFSM4VO6JJDA .