gully
commented
3 years ago Ok, I've made a minimal working IGRINS container based off of astropy's specutils, and added a trivial custom method .normalize() to show what I have in mind. Here is a simple example:
from muler.igrins import IGRINSSpectrum
# Read in the spectrum
file = 'data/SDCH_20201225_1234.spec_a0v.fits'
spec = IGRINSSpectrum(file=file, order=10)
# Normalize method as a demo
spec = spec.normalize()We can add lots of methods to abstract-away the complexity of routine operations. To fuel the imagination, consider the prospect of vaporware methods like:
.mask_telluric_regions( ).refine_wavelength_calibration(method='telluric').stitch_echelle_orders( ).remove_outliers( )
The IGRINSSpectrum object is an instance of specutils' Spectrum1D, and the flux and wavelength attributes are astropy quantity objects, which are subclasses of familiar numpy arrays:
assert isinstance(spec, Spectrum1D) # True
assert isinstance(spec.flux, np.ndarray) # TrueIn the guts of IGRINSSpectrum, it manually deals with fits files for simplicity, but if specutils were chosen as the right building block, one would take the time to write a custom data loader. An open question is whether Spectrum1D is the strategic building block, instead of say pandas dataframes. For example, lightkurve originally just defined the time and flux numpy arrays themselves, but has since moved to build off of astropy TimeSeries, with a significant cost to the migration process. So we'd like to make the right choice from the beginning.
My hunch is that Spectrum1D objects are the right building block, since they're specifically designed for astronomical applications. I mostly see upsides to specutils, but it's worth mentioning the conceivable demerits:
Units-- Sometimes astropy units can get in the way, and weird gotchas pop-up when one is used to working with bare numpy arrays and dealing with the units themselves. I like units, but they can occasionally cause more harm than good.
Bugs and warnings-- So far specutils has been really verbose about warnings and various deprecations. Lots of warnings like this detract from the simplicity we're going for.
Performance/robustness tradeoff-- When I divided flux by a scalar, it somehow involved scipy interpolate. There's no need to bring interpolation into the equation when broadcasting a numpy scalar to a vector. It was hard to trace down where this interpolation warning is coming from because there are multiple inheritance mixins here and there. I think the aim of this module is to make it easy to access data. If performance is needed we can make individual methods more performant, or offer keyword argument toggles, like method='precise'.
kfkaplan
The problem to solve
You have just received data from the IGRINS spectrograph and you want to start science. You may get handed plp-reduced data from the observatory facility or from a collaborator, or self-reduced with
plp. When you examine the data you may notice some remaining instrumental artifacts: incomplete telluric cancellation artifacts remain. These data artifacts hinder your science progress. Or maybe you want to apply a barycentric correction based on telescope pointing information in the FITS header, but you're not sure which FITS header columns to use when multiple are available. You may want to normalize the spectrum, or populate the spectrum into a pandas dataframe, or estimate a coarse radial velocity based on a template. All of these operations are relatively routine, but their application, order, and algorithm choice may depend on the science case, and therefore they cannot be built into a default pipeline.Instead, IGRINS users develop bespoke methods for their science application of interest. This code development is relatively costly in time, as each user may "re-invent the wheel" for common operations that fellow IGRINS users have already solved. We wish to avoid the "re-inventing the wheel" phenomenon through encouraging code re-use, and a shared framework for IGRINS data analysis.
The solution and project scope
Here we propose a Python Package for plp-reduced IGRINS data. This package will introduce relevant Python containers that possess common attributes and routine methods. The inspirations for this project are principally lightkurve and specutils. Specifically, this project will encourage an API design, rather than a commandline/script-centric design. The scope of the project is limited to 1D echelle spectra, not the 2D raw frames, which are the domain of the plp. A key idea for the proposed package is that the
plpessentially introduced a standard--- all IGRINS data have (roughly) the same attributes and data columns. This package will take advantage of that standardization, and can optionally accommodate data from previous plp version that disobey the current standard. The project will initially begin with local data only, with prospects in the future for querying and populating data from remote databases.The key priorities of this package are:
Project Architecture
I propose an architecture closely matching that of
lightkurve, which has become an exemplar for ease-of-use within modern astronomical data analysis. The key leap of the imagination is to change from time-series analysis to spectral analysis. To first approximation that's merely by replacing time with wavelength. In practice, several echelle spectroscopy-specific concepts may make the mental mapping a bit subtle to architect. We also want to encourage the reuse of code fromspecutils.Some architecture decisions to consider: