search

desihub / desispec

DESI spectral pipeline
BSD 3-Clause "New" or "Revised" License
36 stars 24 forks source link

emlinefit: do not fit ebv >= ebvmax=2 spectra #2195

Closed araichoor closed 6 months ago

araichoor commented 6 months ago

This PR addresses https://github.com/desihub/desispec/issues/1985 and https://github.com/desihub/desispec/issues/2186.

The workaround is just to not fit the few spectra with EBV >= 2. Based from the discussion in https://github.com/desihub/desispec/issues/2186, this should concerns only few spectra close to the Galactic plane.

This is controlled by a new argument in desispec.io.emline.read_emlines_inputs(), ebvmax, which defaults to 2. read_emlines_inputs() will set ivar=0 for all pixels of spectra with ebv >= ebvmax.

Once read, those values are then feeded to desispec.emlinefit.get_emlines(): https://github.com/desihub/desispec/blob/fe5a567126659dea873c55c1d9afc864eb6dcf7d/py/desispec/scripts/emlinefit.py#L139-L163 For those spectra, as no pixel is considered as valid, the code will not perform the fit (note that such a case could already happen if the code does not find enough "good" pixels in the region of interest). Spectra which are not fitted have -99 and np.nan values in the output table .

I chose this approach, i.e. do the modification in the spectra reading step, because the ebv quantity is not known in get_emlines(), and I thought it would be better to not change the arguments of that function.

With this PR, the two coadds mentioned in the issues run in ~30s on the login node. Here s one example:

desi_emlinefit_afterburner --coadd /global/cfs/cdirs/desi/spectro/redux/iron/tiles/cumulative/82406/20211118/coadd-5-82406-thru20211118.fits  --redrock /global/cfs/cdirs/desi/spectro/redux/iron/tiles/cumulative/82406/20211118/redrock-5-82406-thru20211118.fits --output /global/cfs/cdirs/desi/users/raichoor/tmpdir/emline-5-82406-thru20211118.fits
[...]
INFO:emlinefit.py:183:read_emlines_inputs: Dealing with 500 spectra
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2882312762455753032 (EBV=2.75 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2305843012414221480 (EBV=2.85 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=-824065148 (EBV=2.41 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=-824065174 (EBV=3.30 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2305843012414233746 (EBV=2.82 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2305843012414223074 (EBV=2.02 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=-824065161 (EBV=3.96 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2305843012414236437 (EBV=3.94 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2305843012414230129 (EBV=2.94 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2305843012414225328 (EBV=2.19 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2305843012414226888 (EBV=2.11 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2305843012414220503 (EBV=2.66 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2305843012414228951 (EBV=2.55 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2305843012414221562 (EBV=2.72 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2305843012414238605 (EBV=2.50 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2305843012414240844 (EBV=3.26 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2882312762455752711 (EBV=3.95 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2305843012414232726 (EBV=3.24 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2305843012414224787 (EBV=2.08 >= 2.0)
WARNING:emlinefit.py:259:read_emlines_inputs: set ivar=0 for all pixels of TARGETID=2305843012414220513 (EBV=2.52 >= 2.0)
INFO:emlinefit.py:264:read_emlines_inputs: we set ivar=0 for 20/500 spectra with EBV >= 2.0
INFO:emlinefit.py:149:main: 0.4s    read_done   TIMESTAMP=2024-03-14T17:46:53.533
INFO:emlinefit.py:163:main: 33.2s   fit_done    TIMESTAMP=2024-03-14T17:47:26.328
INFO:emlinefit.py:179:main: 33.2s   write_done  TIMESTAMP=2024-03-14T17:47:26.336
INFO:emlinefit.py:203:main: 33.2s   done    TIMESTAMP=2024-03-14T17:47:26.336