Roman-Supernova-PIT / diff-img

1 stars 0 forks source link

Refactoring of sfft_and_animate necessary #18

Closed laldoroty closed 3 months ago

laldoroty commented 3 months ago

Per this issue: https://github.com/Roman-Supernova-PIT/phrosty/issues/15

laldoroty commented 3 months ago

Current version of code gets through one science image, and then crashes. For now, the resulting decorrelated difference image looks like this (sorry it's huge): image

All the things in it look like they've got a dipole oriented in the same direction, which tells me that the two images are slightly misaligned.

The log for the second image is:

*************************************************************
R062 35088 2
/hpc/group/cosmology/lna18/miniconda3/envs/repeatability/lib/python3.11/site-packages/erfa/core.py:133: ErfaWarning: ERFA function "d2dtf" yielded 1 of "dubious year (Note 5)"
  warn(f'ERFA function "{func_name}" yielded {wmsg}', ErfaWarning)
Savepath PSF:
/work/lna18/imsub_out/psf/psf_7.551093401915147_-44.80718106491529_R062_35088_2.fits

Path to science PSF:
/work/lna18/imsub_out/psf/psf_7.551093401915147_-44.80718106491529_R062_35088_2.fits

Savepath PSF:
/work/lna18/imsub_out/psf/psf_7.551093401915147_-44.80718106491529_R062_6_17.fits
MeLOn CheckPoint: NO IMAGE ZOOM!
MeLOn CheckPoint: Modify WCS to adapt to the Rotated Frame.
MeLOn CheckPoint: Run SWarp Command ... 
 cd /tmp/PYSWarp_kihzu2qg && swarp /tmp/tmp_n2zr5x2/original_image.fits -IMAGEOUT_NAME /tmp/PYSWarp_kihzu2qg/original_image.tmp_resamp.fits -WEIGHTOUT_NAME /tmp/PYSWarp_kihzu2qg/original_image.tmp_resamp_weight.fits -c /tmp/PYSWarp_kihzu2qg/PYSWarp_4hek9rhuswarp_config/PYSWarp.swarp
----- SWarp 2.38.0 started on 2024-07-24 at 18:21:07 with 62 threads

> Creating NEW output image ...
> WARNING: FITS header data read in /tmp/PYSWarp_kihzu2qg/original_image.tmp_resamp.head

------- Output File original_image.tmp_resamp.fits:
    "no ident"  WEIGHTED  EXT. HEADER  201x201  32 bits (floats)
    Center: 12:00:00.00 +00:00:00.0   3.35'x3.35'  Scale: 1 ''/pixel
    Gain: 0 e-/ADU   Flux scaling (astrom/photom): 1 X / 1 X

-------------- File original_image.fits:
    "no ident"  unweighted  no ext. header  201x201  32 bits (floats)
    Center: 12:00:00.00 +00:00:00.0   3.35'x3.35'  Scale: 1 ''/pixel
    Gain: 1 e-/ADU   Flux scaling (astrom/photom): 1 X / 1 X
    Background: -2.258311e-11   RMS: 4.976979e-10

-------------- Co-adding frames            
Maximum overlap density: 1 frame
> All done (in 0.1 s)

Path to template PSF:
/work/lna18/imsub_out/psf/rot_psf_7.551093401915147_-44.80718106491529_R062_6_17.fits.fits

The following images overlap sufficiently:
/work/lna18/imsub_out/align/align_skysub_Roman_TDS_simple_model_R062_6_17.fits
/work/lna18/imsub_out/skysub/skysub_Roman_TDS_simple_model_R062_35088_2.fits

Path to cross-convolved science image:
/work/lna18/imsub_out/convolved/conv_skysub_Roman_TDS_simple_model_R062_35088_2.fits
Path to cross-convolved template image:
/work/lna18/imsub_out/convolved/conv_align_skysub_Roman_TDS_simple_model_R062_6_17.fits

Traceback (most recent call last):
  File "/hpc/group/cosmology/lna18/rsim_photometry/sfft_and_animate.py", line 251, in <module>
    parse_and_run()
  File "/hpc/group/cosmology/lna18/rsim_photometry/sfft_and_animate.py", line 247, in parse_and_run
    run(args.oid, args.band, args.n_templates, args.verbose)
  File "/hpc/group/cosmology/lna18/rsim_photometry/sfft_and_animate.py", line 178, in run
    sfft(ra,dec,band,in_tab,template_tab,verbose=True)
  File "/hpc/group/cosmology/lna18/rsim_photometry/sfft_and_animate.py", line 136, in sfft
    diff, soln = difference(sci_conv,temp_conv,sci_psf_path,t_psf_path,savename=diff_savename,backend='Numpy',force=True)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/hpc/group/cosmology/lna18/phrosty/phrosty/imagesubtraction.py", line 317, in difference
    sci_data = fits.getdata(scipath).T
               ^^^^^^^^^^^^^^^^^^^^^
  File "/hpc/group/cosmology/lna18/miniconda3/envs/repeatability/lib/python3.11/site-packages/astropy/io/fits/convenience.py", line 220, in getdata
    hdulist, extidx = _getext(filename, mode, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/hpc/group/cosmology/lna18/miniconda3/envs/repeatability/lib/python3.11/site-packages/astropy/io/fits/convenience.py", line 1121, in _getext
    hdulist = fitsopen(filename, mode=mode, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/hpc/group/cosmology/lna18/miniconda3/envs/repeatability/lib/python3.11/site-packages/astropy/io/fits/hdu/hdulist.py", line 222, in fitsopen
    return HDUList.fromfile(
           ^^^^^^^^^^^^^^^^^
  File "/hpc/group/cosmology/lna18/miniconda3/envs/repeatability/lib/python3.11/site-packages/astropy/io/fits/hdu/hdulist.py", line 486, in fromfile
    return cls._readfrom(
           ^^^^^^^^^^^^^^
  File "/hpc/group/cosmology/lna18/miniconda3/envs/repeatability/lib/python3.11/site-packages/astropy/io/fits/hdu/hdulist.py", line 1169, in _readfrom
    fileobj = _File(
              ^^^^^^
  File "/hpc/group/cosmology/lna18/miniconda3/envs/repeatability/lib/python3.11/site-packages/astropy/io/fits/file.py", line 218, in __init__
    self._open_filename(fileobj, mode, overwrite)
  File "/hpc/group/cosmology/lna18/miniconda3/envs/repeatability/lib/python3.11/site-packages/astropy/io/fits/file.py", line 641, in _open_filename
    self._file = open(self.name, IO_FITS_MODES[mode])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/work/lna18/imsub_out/convolved/conv_skysub_Roman_TDS_simple_model_R062_35088_2.fits'

Odd, because the same path clearly printed above. I didn't have "force" toggled True on the cross convolve function, so I'll change that to True and re-run tomorrow.

laldoroty commented 3 months ago

I didn't have "force" toggled True on the cross convolve function, so I'll change that to True and re-run tomorrow.

Fixed the crash! Now have to find the alignment issue.

laldoroty commented 3 months ago

Actually, I think that improved, but did not fix, the alignment issue. It must have been pulling from old PSFs that were incorrect. image

laldoroty commented 3 months ago

100x100 around the SN looks decent (this is the first image it shows up, so shouldn't be super bright yet:

image

laldoroty commented 3 months ago

NOT! BAD! But I need to figure out why it's yeeting all those images around peak.

image

laldoroty commented 3 months ago

The pipeline runs, so I am going to request review to close this, and open a new issue for the light curve.