Added the capability for multple arc files, with linear interpolation

[mikeireland]

of wavelength scales in-between the time of each arc. Also prevented a crash if the flat cleanup step can't be run (e.g. for any stellar mode data).

[ghost]

That looks good. I will most likely add a few minor things when merging it:

• check if the two arc frames are on either side of the exposure, and revert to one arc only and/or issue a warning if this is not the case. Do you think we should allow extrapolation of the arc solution ? I feel that this could lead to many disasters, and it might be better to simply revert to using one arc only, when they do not bracket an exposure. I.e. people who know about this will do it properly when observing, and people who don't won't be affected. What do you think ?
• There used to be an issue with the recorded times of the exposures - sometimes, they would be registered at the end of the read-out phase, rather than the beginning of the integration. This is not too dramatic in itself, as (as far as I know) it occurs once in a while - but given that this upgrade is designed for high wavelength precision, we should make sure that the times we use are as accurate as possible.
• using the filename to get the time is a bad idea. I, for example, first rename all my exposures to b???.fits and r???.fits because I hate having to type all their names explicitly in the meta data file. We should use the information from the header, either the 'FILENAME' one, or probably better and faster, the Modified Julian Date of the acquisition ('MJD-OBS'), so we can directly interpolate between numbers.

Does this sound reasonable to you ?

[mikeireland]

All agreed. The arcs should not be extrapolated - I'm sure that if you used a Gaussian process to find the optimal wavelength solution the end result would be close to a linear interpolation or nearest arc if not possible. Note that the times are not in the headers of the wsol fits files, so this also has to be transferred from the headers of the original files. My minimum standard for the dodgy "time from filename" change was that the code should work and shouldn't break existing analyses (which only use one arc).

[ghost]

Good point about the header of the wsol files. I'll track their creation down, and make sure that (at least) the MJD is included when they are created. Will probably get around doing this early next week, and will then update the repository accordingly.

[mikeireland]

Hi Fred - any progress on your additions here to my pull request? I don't want to end up with a diverging fork if possible.

[ghost]

Sorry, thesis submission got in the way. I updated the reduce_red/blue_data.py files in the master branch following your pull request, and implemented the points raised in the discussion above:

• only interpolate if the arcs surround the Science frame
• only interpolate if only two arcs are present (otherwise, a sign that the user doesn't understand what is happening)
• use date/times from the header, and rely on the datetime module to calculate time differences.

I did a quick test and it seems to work fine (i.e. no error thrown). I do not have time to go deeper and perform more extended tests just now, so, please let me know if you spot something odd/wrong when you try the new reduction recipes.

Cheers !

P.S.: closing pull request. I think it'll be best for all new (possible) bugs associated with this update to be added to the 'issues' page.