Compare Jura Redrock QSO redshifts to https://arxiv.org/abs/2308.15586

[sbailey]

Wu & Shen abstract from https://arxiv.org/abs/2308.15586:

The Dark Energy Spectroscopic Instrument (DESI) survey will provide optical spectra for ∼3 million quasars. Accurate redshifts for these quasars will facilitate a broad range of science applications. Here we provide improved systemic redshift estimates for the ∼95k quasars included in the DESI Early Data Release (EDR), based on emission-line fits to the quasar spectra. The majority of the DESI pipeline redshifts are reliable. However, ∼19% of the EDR quasars have pipeline redshifts that deviate from our new redshifts by >500kms−1. We use composite quasar spectra to demonstrate the improvement of our redshift estimates, particularly at z>1. These new redshifts are available at this https URL.

That was based upon the Fuji/EDR QSO templates from BOSS. Re-evaluate the situation with Jura, which has @abrodze's updated LOZ/HIZ QSO templates, including the post-Iron HIZ templates.

[abrodze]

The Wu & Shen catalog contains ~94,000 entries but I compare only 18,673 to their Jura counterpart here. Of the remaining entries, they either could not be matched to DESI observations or had erroneous redshift estimates (i.e. z>100 or z<0).

The authors used the EDR healpix redshift catalog without afterburner information for their analysis, so I compared their redshifts to the non-afterburner informed Jura redshifts. They also restricted to ZWARN==0 and SPECTYPE=QSO in EDR.

Wu & Shen report the following redshift offsets (km/s) between EDR and their estimate: 500 < |dv| < 1000 - 13.2% 1000 < |dv| < 1500 - 3.2% |dv| > 1500 - 2.2%

Directly compared to jura redshifts without data cuts, I find: 500 < |dv| < 1000 - 11.8% 1000 < |dv| < 1500 - 3% 1500 < |dv| < 3000 - 2% |dv| > 3000 - 3.6%

Excluding objects that are no longer classified as QSO in jura (the majority of |dv|>3000 km/s instances): 500 < |dv| < 1000 - 12.3% 1000 < |dv| < 1500 - 3.1% 1500 < |dv| < 3000 - 2% |dv| > 3000 - 1.2%

Additionally excluding objects that now have ZWARN != 0 in Jura: 500 < |dv| < 1000 - 12.2% 1000 < |dv| < 1500 - 3.0% 1500 < |dv| < 3000 - 1.9% |dv| > 3000- 1.0%

The majority of remaining |dv| > 3000 km/s are cases of line confusion, see below. The Wu catalog redshift took the EDR redshift as a prior so if EDR was catastrophically wrong, there is a high chance the Wu redshift failed as well. As such, some of these off diagonal points may actually be good Jura redshifts. It is likely QN would help us in the case of true line confusion, but it'd be interesting to see if (at least some of) these are actually Redrock moving in the correct direction.

There also appears to be a slight redshift dependency in the velocity offset (plot below excludes |dv| > 10000 km/s):

For context, the redshifts from the Wu catalog come from a weighted average of the prominent emission line redshifts. Above z>2, their redshift relies on the CIII], CIV, SiIV, and Lya complexes. The “average intrinsic shifts of CIV and SIV” are accounted for in their work when estimating Z_CIV(SIV) -- I assume using Shen+16 values (?). The spread of this shift however, is quite large and tailed blue (ex. Fig 2 of Shen+16) so the correction is not perfect.

The magnitude of the average offset of Jura from the Wu catalog redshifts at z>3 appears to be consistent with expected CIV/SIV velocity shifts. This would suggest our templates are not entirely robust against line shifting after losing the low ionization lines. Though, it is important to note the quasar numbers rapidly drop with z>3 in this comparison.

[abrodze]

All objects from the Wu & Shen EDR catalog have been successfully matched to the Jura healpix catalog after addressing some corruption in the file.

excluding SPECTYPE!=QSO in Jura: 500 < |dv| < 1000 - 11.6% 1000 < |dv| < 1500 - 2.9% 1500 < |dv| < 3000 - 1.8% |dv| > 3000- 1.1%

additionally excluding objects that now have zwarn!=0 in Jura: 500 < |dv| < 1000 - 11.5% 1000 < |dv| < 1500 - 2.8% 1500 < |dv| < 3000 - 1.7% |dv| > 3000 - 1.0%

Line confusion between the catalogs is still the dominant source of disagreement for Jura SPECTYPE='QSO' objects. Using @dylanagreen's line confusion stacking technique, it appears the Jura redshift is correct in the majority of cases and/or the spectrum has calibration errors. The exception is CIV->Lya confusion, where the Wu catalog redshift is more often correct. Line confusion cases are likely a result of the Wu & Shen redshift using an incorrect EDR redshift as prior.

The redshift dependency seen in the subsample above is the same with the full catalog; however when compared to EDR the situation is much improved. The rest frame of the Jura templates is more consistent at all redshifts than the EDR templates. The positive impact of the optical depth correction in Jura is also clear at z>2. The average shift from Z_Jura to Z_WU remains < ~175 km/s up to z=4. As stated in the last comment, the persisting z dependency likely comes from the QSO templates being less robust against the velocity shift of high ionization lines owing to a combination of (1) the higher luminosities (and therefore greater line shifts) at the high redshift end of the quasar sample and (2) the lack of any of the more stable, low ionization lines for reference.