Add additional flexibility to the wavelength solution

bjfultn commented 8 years ago

Add additional flexibility to the wavelength solution within grand. Spline fit with nodes at the LSF locations?

bjfultn commented 8 years ago

@JeffValenti send wavelength solution used for the simulations to Jay.

JeffValenti commented 8 years ago

The wavelength solution in hires.wave is a 4th-order polynomial in the pixel range [20:4000]. There is a wavelength discontinuity between this central region and the edges at [0:19] and [4001:4020]. The simulated data replaces wavelengths in these edge regions with values obtained by extrapolating the 4th-order polynomial in the central region. Thus, the simulated wavelength scale can be exactly modeled by grand.F, which currently uses a 4th-order polynomial.

JeffValenti commented 8 years ago

I added a dump= keyword to simobs_set.pro and simobs_one.pro. Specifying dump=1 causes simobs_one.pro to write out pixel level information (including wavelength scale) for each observation and echelle order. Specifying dump=2 causes simobs_one.pro to write out supersampled information for each observation and echelle order. I reran my most recent simulation with dump=2, creating a full set of auxiliary files:

simobs_one,'sun','keck-nso-52.5','jay2015','hires',-21766.49,0,'sun.018',seed=200,dump=2

I used the same pseudo-random number seed as last time, so the simulated spectra are identical. The new files are in 13_sun_nso525_jay15_hires_30_100_0_0.tar.gz [1 GB] on Box.

We can compare the simulated wavelength scale for each observation and order with the output wavelength scale that BJ has already determined using iGrand. See Issue #3.

JeffValenti commented 8 years ago

@JayVB and I just discussed this issue. Currently, the wavelength solution is a spline curve with 5 nodes across an echelle order that is immediately fitted with a 4th-order polynomial. Currently, grand.F uses linear least squares to solves for wavelength node values by linearizing the RMS residual of the fit with respect to wavelength node values. Jay will:

Modify grand.F to use a spline curve with the same node locations as the LSF. This means wavelength will have 9 nodes instead of 5 nodes. The interpolating/extrapolating function will be at least quadratic all the way to the edge of the order. Jay plans to determine wavelength node values locally, rather than globally as grand.F currently does.
Comment out or remove the step that fits the wavelength solution with a 4th-order polynomial. The wavelength solution will be much more flexible.

See Issue #21 for a description of the new wavelength inputs, outputs, and switches that Jay will implement in grand.F.

JayVB commented 8 years ago

I have made some progress over the last few days on my main assignment (this issue):

I have looked into how the LSF is interpolated between its nodes and the routine does indeed use a spline; it may be worthwhile doing things just linearly so that problems at the edges can be isolated to the edges.
I have revamped the WLS solution and now instead of using (essentially) a 4th order (5 parameter) WLS, it uses the exact same 9 spline nodes it uses for the LSF, namely the first at i=21 and the last at i=4000 and seven evenly spaced between.
It does look like there is WLS structure in between the 5-point nodes, and the LSF was bouncing between them. Fixing this will be good.
It doesn't always reach "convergence" at the very edges, but it's probably fine for what it does. I don't do a least-squares solution for the WLS anymore, which could easily be subject to outliers. The approach I use now is much more robust and it seems to do a pretty good job.
I have some new code that runs. It will be very very interesting to see if it avoids the problems in orders 4 and 9 that we were seeing before. I'm optimistic.
I was using one of BJ's old OBSLIST files and there were a couple template observations in there. Jeff informed me that BJ had culled the list and put out a new list, but I didn't know that. I took the opportunity to have the code identify, during the wavelength-solution procedure, the observations that didn't yield a good wavelength solution. It collects this list of bad observations and if the list is not a null list, it prints out a list of the offending observations in the OBSLIST file and halts. If there are no problem observations, it should continue as normal.
Remaining to be done is the facility of inputting the WLS from a file. That should be trivial to implement. I now output the WLS at the 9-node points, which is how it is now specified.
Let me know how to proceed; I could send BJ (and Jeff) privately the code that does this, so that we can see whether there are indeed the hoped-for improvements in orders 4 and 9 (and elsewhere with the improved, properly-centered interpolation), then if it does improve things, I could commit it as a new version of grand....

bjfultn commented 8 years ago

Update: this has been implemented, but its not performing very well. @JayVB is working on it and I will leave this issue open for now

bjfultn commented 8 years ago

From our 6/28 meeting: Jay has implemented his "old fitter" with the new wavelength solution prescription and it appears to be working.

@JayVB will look into the "kinks" in the input vs. output wavelength solution.

bjfultn commented 7 years ago

Closing this along with issue #38

California-Planet-Search / grandsol

Add additional flexibility to the wavelength solution #22