Closed rongpu closed 2 weeks ago
The worst one, 1008, was masked from Y1 analysis (list here https://data.desi.lbl.gov/desi/survey/catalogs/Y1/LSS/iron/unique_badfibers.txt). (Does anyone remember if we diagnosed its issue?) Others showing up here that were masked from Y1 include: 466, 651, 3038, 3994, 4321, 4507, 4733, 4891 Is it possible that these have actually been made better in Jura?
There is certainly a significant overlap with the issues that we found in iron. We did attempt to fix 1008 and 3994 in iron, but apparently it did not work (see https://github.com/desihub/desispec/issues/1946#issuecomment-1418297686) and I don't think we ever looked into them in iron.
Some complementary plots are here (where what was masked in Y1 has Xs on the points): https://data.desi.lbl.gov/desi/survey/catalogs/DA2/LSS/jura-v1/plots/
Individual fiber issues have been handled in other tickets. Closing this one for Kibo.
In order to separate the purely software issues from the new hardware issues in the jura data quality checks/investigations, here I have done a direct redshift comparison between Y1 (iron) and Y3 (jura). I am the cumulative zcatalogs, and I match both FIBER and TILEID so that the redshifts have the exact same input spectra. I require FIBERSTATUS==0 and EFFTIME_LRG>800 (or EFFTIME_BGS>160 for BGS), but otherwise no redshift quality cuts are applied.
I define a redshift outlier ("n_fail" and "frac_fail" in the tables below) as any object whose redshifts in the two catalogs differ by more than 0.003333*(1+Z_y1), or 1000 km/s.
BGS_ANY: Average outlier rate: 0.18% 17 fibers with >1.5% outlier rate:
LRG: Average outlier rate: 0.20% 19 fibers with >2% outlier rate:
ELG: Average outlier rate: 4.20% 17 fibers with >10% outlier rate:
The histograms of the per-fiber outlier fractions with the threshold marked by the dashed line:
![image](https://github.com/user-attachments/assets/ca9c7a93-a32f-4d07-b8fb-acea6865c5e6)