search

sdss / lvmdrp

Local Volume Mapper (LVM) Data Reduction Pipeline
BSD 3-Clause "New" or "Revised" License
2 stars 0 forks source link

lvmdrp is failing on straylight subtractio in some cases #102

Closed kslong closed 1 month ago

kslong commented 1 month ago

This attempt to run the drp failes

drp run  -m 60203 -e 5013

with the following error:

INFO]: plotting results
[INFO]: writing stray light image to lvm-dstray-r1-00005013.fits
[INFO]: extracting fiber spectra using fiber profile fitting from lvm-lobject-r1-00005013.fits
[INFO]: median shift in fibers: median_shift = 0.0167+/-std_shift = 0.3023 pixels for mjd = 60203, expnum = 5013, camera = 'r1'
[INFO]: plotting fiber thermal shifts
[INFO]: writing extracted spectra to lvm-xobject-r1-00005013.fits
[INFO]: using fiducial master calibration frames for r2 at masters_path = '/Users/long/Projects/lvm_data/sas/sdsswork/lvm/sandbox/calib/60177'
[INFO]: loading frame from /Users/long/Projects/lvm_data/sas/sdsswork/lvm/spectro/redux/1.0.2dev/0011XX/11111/60203/ancillary/lvm-dobject-r2-00005013.fits
WARNING: FITSFixedWarning: The WCS transformation has more axes (2) than the image it is associated with (0) [astropy.wcs.wcs]
[INFO]: Using Fiducial Platescale = 112.37 "/mm
[INFO]: writing RA,DEC to slitmap in image 'lvm-dobject-r2-00005013.fits'
WARNING: VerifyWarning: Card is too long, comment will be truncated. [astropy.io.fits.card]
[INFO]: using image lvm-dobject-r2-00005013.fits for stray light subtraction
[INFO]: median filtering image along dispersion axis with a median filter of width 101
[INFO]: using centroids trace lvm-mtrace-r2.fits to mask fibers
[INFO]: masking fibers with an aperture of 15 pixels
[INFO]: selecting top 5 and bottom 5 rows of the CCD
[INFO]: fitting spline with smoothing = 400 to the background signal along cross-dispersion axis
[INFO]: interpolating spline fit in 4086 columns
[ERROR]: Failed to reduce science frame mjd 60203 exposure 5013: cannot reshape array of size 0 into shape (0,newaxis)
Traceback (most recent call last):
  File "/Users/long/SDSS/lvmdrp/python/lvmdrp/main.py", line 1895, in run_drp
    science_reduction(expnum, use_fiducial_master=True, clean_ancillary=clean_ancillary,
  File "/Users/long/SDSS/lvmdrp/python/lvmdrp/main.py", line 1686, in science_reduction
    subtract_straylight(in_image=dsci_path, out_image=lsci_path, out_stray=lstr_path,
  File "/Users/long/SDSS/lvmdrp/python/lvmdrp/functions/imageMethod.py", line 2546, in subtract_straylight
    img_fit = img_median.fitSpline(smoothing=smoothing, use_weights=use_weights, clip=(0.0, None))
  File "/Users/long/SDSS/lvmdrp/python/lvmdrp/core/image.py", line 2078, in fitSpline
    f = interpolate.interp1d(x_pixels[~masked_columns], models[:, ~masked_columns], axis=1, bounds_error=False, fill_value="extrapolate")
  File "/Users/long/anaconda3/envs/xlvmdrp/lib/python3.10/site-packages/scipy/interpolate/_interpolate.py", line 543, in __init__
    self._y = self._reshape_yi(self.y)
  File "/Users/long/anaconda3/envs/xlvmdrp/lib/python3.10/site-packages/scipy/interpolate/_polyint.py", line 113, in _reshape_yi
    return yi.reshape((yi.shape[0], -1))
ValueError: cannot reshape array of size 0 into shape (0,newaxis)

The problem is in core/image.py as indicated.

What is happening is that x_pixels[~masked_columns] has zero length, which not surprisingly results in a problem.

For a number of the exposures I was trying to analyze this same issue arose.

kslong commented 1 month ago

I made the following change the code and this particular exposure ran to completion

        # interpolate failed columns if requested
        masked_columns = numpy.count_nonzero((models == 0)|numpy.isnan(models), axis=0) >= 0.1*self._dim[0]
        xdummy= numpy.arange(self._dim[1])
        if len(xdummy[~masked_columns])>0 and interpolate_missing and masked_columns.any():