Difference between SDSS and BOSS spectra

[yanliang-astro]

@pmelchior and I found that sometimes the SDSS and BOSS spectra look systematically different even for the same target. We suspected that this might be a result of different calibration procedures, but we are not sure. @changhoonhahn @agoulding is there a fundamental reason why SDSS and BOSS spectra are different?

See below for an example: SDSS: 401-51788-338 BOSS: 3639-55205-635

[agoulding]

The zeroth-order answer is yes, the spectra can be different. Now, as to why (and precisely, how) they are different is dependent on the object. If the object contains an active nucleus or is a star or a SNe has gone off in the galaxy, then these are variable or transient phenomena and can change the shape of the continuum and/or the emission/absorption strengths. These are physical changes to the spectra and for some objects are expected and interesting to observe. However, I can think of another very obvious thing that can change the spectra as you have them...data reduction. Flux calibrated spectra rely on our ability to do precisely that -- flux calibrate them. If the conditions are particularly poor, if a different fiber is used, if throughput isn't well characterized, if the sky is under/over subtracted, if a different standard star is used etc etc etc etc these can all change the final reduced data product. Indeed, in your example spectra here, we have an old red and dead luminous red galaxy. There is nothing of any note in this spectrum that suggests a physical phenomenon has occurred to change the spectrum. As such, I would put this down to mis-match of calibration between SDSS and BOSS, and so a data taking/reduction issue.

[pmelchior]

Yeah, I had a chat with Michael about this today. He suspects the spectrophotometric calibration to be a likely culprit. For the SDSS redshift fitter, they take out a low-order polynomial from their best fitting template to account for that. We could mimic that be adding the same polynomial (with some variation of its parameters) while we're training. This would effectively marginalize over the calibration.

[yanliang-astro]

Thank you both for your advise! I'll implement a low order polynomial to represent the calibration difference.

[pmelchior]

Details about the polynomial are in this paper

[pmelchior]

Closer inspection showed quite some variation of the mismatch between the spectra. Sometimes it's in the lines (which is discussed in #8), other times in continuum properties, which we'll focus on here.

To determine how much the calibration really does, let's use a model that is trained on SDSS exclusively. Then use a spectrum in the intersection set of SDSS and BOSS and run it through the SDSS autoencoder. Then take a BOSS spectrum and linearly interpolate it to the SDSS spectral bins. Now both spectra are at the same observed wavelengths, so you can compute $\Delta f = f\mathrm{BOSS} - f\mathrm{SDSS}$ vs $\lambda_\mathrm{SDSS}$. To get a calibration function, use a Gaussian Process regression from GPytorch. For the error estimate on each data point, you can add the variances from each spectrum (where the weights are set as inverse variance). The will account for both the noise and the intrinsic variation of the calibration.

I'd start with a few individual spectra to see how the calibration comes out. But ultimately, we want to see how the calibration varies over the entire surveys, so it'll be best to combine $\Delta f$ vs $\lambda$ samples from many different spectra.