search

Roman-Supernova-PIT / phrosty

Basic package for photometry on the RomanDESC sims.
MIT License
4 stars 2 forks source link

Failure mode of pipeline: Empty filename #33

Open laldoroty opened 1 month ago

laldoroty commented 1 month ago

Some files are failing like this: 

The following have been reloaded with a version change:
  1) conda/Mambaforge-23.1.0-1 => conda/Miniconda3-py311_23.11.0-2

The following have been reloaded with a version change:
  1) conda/Miniconda3-py311_23.11.0-2 => conda/Mambaforge-23.1.0-1

20000677
[2024-10-21 13:44:55 - phrosty - ERROR] Sky subtraction subprocess failure: Empty or corrupt FITS file

MeLOn CheckPoint [Roman_TDS_simple_model_H158_53350_11.fits]: Run Python Wrapper of SExtractor!
WARNING: File may have been truncated: actual file length (1458176) is smaller than the expected size (133701120) [astropy.io.fits.file]
[2024-10-21 13:44:56 - phrosty - ERROR] Sky subtraction subprocess failure: list index out of range

MeLOn CheckPoint [Roman_TDS_simple_model_H158_7585_1.fits]: Run Python Wrapper of SExtractor!

MeLOn CheckPoint [Roman_TDS_simple_model_H158_53355_6.fits]: Run Python Wrapper of SExtractor!

MeLOn CheckPoint [Roman_TDS_simple_model_H158_53735_8.fits]: Run Python Wrapper of SExtractor!

MeLOn CheckPoint [Roman_TDS_simple_model_H158_609_12.fits]: Run Python Wrapper of SExtractor!

MeLOn CheckPoint [Roman_TDS_simple_model_H158_7580_8.fits]: Run Python Wrapper of SExtractor!

MeLOn CheckPoint [Roman_TDS_simple_model_H158_614_17.fits]: Run Python Wrapper of SExtractor!

MeLOn CheckPoint [Roman_TDS_simple_model_H158_4509_13.fits]: Run Python Wrapper of SExtractor!

MeLOn CheckPoint [Roman_TDS_simple_model_H158_53355_6.fits]: Run Python Wrapper of SExtractor!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_53350_11.fits]: SExtractor found [3789] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_53350_11.fits]: PYSEx output catalog contains [3789] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_7585_1.fits]: SExtractor found [4312] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_7585_1.fits]: PYSEx output catalog contains [4312] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_53355_6.fits]: SExtractor found [3899] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_53355_6.fits]: PYSEx output catalog contains [3899] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_53735_8.fits]: SExtractor found [3903] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_53735_8.fits]: PYSEx output catalog contains [3903] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_609_12.fits]: SExtractor found [3945] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_609_12.fits]: PYSEx output catalog contains [3945] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_7580_8.fits]: SExtractor found [3929] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_7580_8.fits]: PYSEx output catalog contains [3929] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_614_17.fits]: SExtractor found [4056] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_614_17.fits]: PYSEx output catalog contains [4056] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_4509_13.fits]: SExtractor found [4240] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_4509_13.fits]: PYSEx output catalog contains [4240] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_53355_6.fits]: SExtractor found [3899] sources!
MeLOn CheckPoint [Roman_TDS_simple_model_H158_53355_6.fits]: PYSEx output catalog contains [3899] sources!
/global/homes/l/laldorot/.conda/envs/diff/lib/python3.11/site-packages/erfa/core.py:133: ErfaWarning: ERFA function "d2dtf" yielded 1 of "dubious year (Note 5)"
  warn(f'ERFA function "{func_name}" yielded {wmsg}', ErfaWarning)
/global/homes/l/laldorot/.conda/envs/diff/lib/python3.11/site-packages/erfa/core.py:133: ErfaWarning: ERFA function "d2dtf" yielded 1 of "dubious year (Note 5)"
  warn(f'ERFA function "{func_name}" yielded {wmsg}', ErfaWarning)
/global/homes/l/laldorot/.conda/envs/diff/lib/python3.11/site-packages/erfa/core.py:133: ErfaWarning: ERFA function "d2dtf" yielded 1 of "dubious year (Note 5)"
  warn(f'ERFA function "{func_name}" yielded {wmsg}', ErfaWarning)
/global/homes/l/laldorot/.conda/envs/diff/lib/python3.11/site-packages/erfa/core.py:133: ErfaWarning: ERFA function "d2dtf" yielded 1 of "dubious year (Note 5)"
  warn(f'ERFA function "{func_name}" yielded {wmsg}', ErfaWarning)
/global/homes/l/laldorot/.conda/envs/diff/lib/python3.11/site-packages/erfa/core.py:133: ErfaWarning: ERFA function "d2dtf" yielded 1 of "dubious year (Note 5)"
  warn(f'ERFA function "{func_name}" yielded {wmsg}', ErfaWarning)
/global/homes/l/laldorot/.conda/envs/diff/lib/python3.11/site-packages/erfa/core.py:133: ErfaWarning: ERFA function "d2dtf" yielded 1 of "dubious year (Note 5)"
  warn(f'ERFA function "{func_name}" yielded {wmsg}', ErfaWarning)
MeLOn CheckPoint: Cuda resampling takes [0.004495 s]
MeLOn CheckPoint: Cuda resampling takes [0.000037 s]
MeLOn CheckPoint: Cuda resampling takes [0.000122 s]
MeLOn CheckPoint: TRIGGER Function Compilations of SFFT-SUBTRACTION!

 --//--//--//--//-- TRIGGER SFFT COMPILATION --//--//--//--//-- 

 ---//--- KerPolyOrder 2 | BGPolyOrder 0 | KerHW [9] ---//--- 

 --//--//--//--//-- EXIT SFFT COMPILATION --//--//--//--//-- 

MeLOn Report: Function Compilations of SFFT-SUBTRACTION TAKES [0.001 s]

                                __    __    __    __
                               /  \  /  \  /  \  /  \
                              /    \/    \/    \/    \
            █████████████████/  /██/  /██/  /██/  /█████████████████████████
                            /  / \   / \   / \   / \  \____
                           /  /   \_/   \_/   \_/   \    o \__,
                          / _/                       \_____/  `
                          |/

                      █████████  ███████████ ███████████ ███████████        
                     ███░░░░░███░░███░░░░░░█░░███░░░░░░█░█░░░███░░░█            
                    ░███    ░░░  ░███   █ ░  ░███   █ ░ ░   ░███  ░ 
                    ░░█████████  ░███████    ░███████       ░███    
                     ░░░░░░░░███ ░███░░░█    ░███░░░█       ░███    
                     ███    ░███ ░███  ░     ░███  ░        ░███    
                    ░░█████████  █████       █████          █████   
                     ░░░░░░░░░  ░░░░░       ░░░░░          ░░░░░         

                    Saccadic Fast Fourier Transform (SFFT) algorithm
                    sfft supported by @LeiHu

                    GitHub: https://github.com/thomasvrussell/sfft
                    Related Paper: https://arxiv.org/abs/2109.09334

            ████████████████████████████████████████████████████████████████

 --||--||--||--||-- TRIGGER SFFT SUBTRACTION --||--||--||--||-- 

 ---||--- KerPolyOrder 2 | BGPolyOrder 0 | KerHW [9] ---||--- 

MeLOn CheckPoint: SFFT-SUBTRACTION Preliminary Steps takes [0.0612s]
/////   a   ///// Read Input Images  (0.0000s)
/////   b   ///// Spatial Polynomial (0.0090s)
/////   c   ///// DFT-8             (0.0499s)

MeLOn CheckPoint: SFFT-SUBTRACTION Establish & Solve Linear System takes [0.4114s]
/////   d   ///// Establish OMG                       (0.1928s)
/////   e   ///// Establish GAM                       (0.0412s)
/////   f   ///// Establish PSI                       (0.0339s)
/////   g   ///// Establish PHI                       (0.0150s)
/////   h   ///// Establish THE & DEL                 (0.0473s)
/////   i   ///// Solve Linear System                 (0.0725s)

 --||--||--||--||-- EXIT SFFT SUBTRACTION --||--||--||--||-- 

 --||--||--||--||-- TRIGGER SFFT SUBTRACTION --||--||--||--||-- 

 ---||--- KerPolyOrder 2 | BGPolyOrder 0 | KerHW [9] ---||--- 

MeLOn CheckPoint: SFFT-SUBTRACTION Preliminary Steps takes [0.0028s]
/////   a   ///// Read Input Images  (0.0000s)
/////   b   ///// Spatial Polynomial (0.0001s)
/////   c   ///// DFT-8             (0.0010s)

MeLOn CheckPoint: SFFT-SUBTRACTION Perform Subtraction takes [0.2554s]
/////   j   ///// Calculate Kab         (0.0657s)
/////   k   ///// Construct DIFF        (0.1897s)

 --||--||--||--||-- EXIT SFFT SUBTRACTION --||--||--||--||-- 

MeLOn Report: SFFT-SUBTRACTION TAKES [0.790 s]
MeLOn CheckPoint: Cuda resampling takes [0.000052 s]
MeLOn CheckPoint: Cuda resampling takes [0.000038 s]
MeLOn CheckPoint: Cuda resampling takes [0.000052 s]
MeLOn CheckPoint: TRIGGER Function Compilations of SFFT-SUBTRACTION!

 --//--//--//--//-- TRIGGER SFFT COMPILATION --//--//--//--//-- 

 ---//--- KerPolyOrder 2 | BGPolyOrder 0 | KerHW [9] ---//--- 

 --//--//--//--//-- EXIT SFFT COMPILATION --//--//--//--//-- 

MeLOn Report: Function Compilations of SFFT-SUBTRACTION TAKES [0.002 s]

                                __    __    __    __
                               /  \  /  \  /  \  /  \
                              /    \/    \/    \/    \
            █████████████████/  /██/  /██/  /██/  /█████████████████████████
                            /  / \   / \   / \   / \  \____
                           /  /   \_/   \_/   \_/   \    o \__,
                          / _/                       \_____/  `
                          |/

                      █████████  ███████████ ███████████ ███████████        
                     ███░░░░░███░░███░░░░░░█░░███░░░░░░█░█░░░███░░░█            
                    ░███    ░░░  ░███   █ ░  ░███   █ ░ ░   ░███  ░ 
                    ░░█████████  ░███████    ░███████       ░███    
                     ░░░░░░░░███ ░███░░░█    ░███░░░█       ░███    
                     ███    ░███ ░███  ░     ░███  ░        ░███    
                    ░░█████████  █████       █████          █████   
                     ░░░░░░░░░  ░░░░░       ░░░░░          ░░░░░         

                    Saccadic Fast Fourier Transform (SFFT) algorithm
                    sfft supported by @LeiHu

                    GitHub: https://github.com/thomasvrussell/sfft
                    Related Paper: https://arxiv.org/abs/2109.09334

            ████████████████████████████████████████████████████████████████

 --||--||--||--||-- TRIGGER SFFT SUBTRACTION --||--||--||--||-- 

 ---||--- KerPolyOrder 2 | BGPolyOrder 0 | KerHW [9] ---||--- 

MeLOn CheckPoint: SFFT-SUBTRACTION Preliminary Steps takes [0.0031s]
/////   a   ///// Read Input Images  (0.0000s)
/////   b   ///// Spatial Polynomial (0.0001s)
/////   c   ///// DFT-8             (0.0010s)

MeLOn CheckPoint: SFFT-SUBTRACTION Establish & Solve Linear System takes [0.3452s]
/////   d   ///// Establish OMG                       (0.0028s)
/////   e   ///// Establish GAM                       (0.0005s)
/////   f   ///// Establish PSI                       (0.0005s)
/////   g   ///// Establish PHI                       (0.0002s)
/////   h   ///// Establish THE & DEL                 (0.0008s)
/////   i   ///// Solve Linear System                 (0.3402s)

 --||--||--||--||-- EXIT SFFT SUBTRACTION --||--||--||--||-- 

 --||--||--||--||-- TRIGGER SFFT SUBTRACTION --||--||--||--||-- 

 ---||--- KerPolyOrder 2 | BGPolyOrder 0 | KerHW [9] ---||--- 

MeLOn CheckPoint: SFFT-SUBTRACTION Preliminary Steps takes [0.0027s]
/////   a   ///// Read Input Images  (0.0000s)
/////   b   ///// Spatial Polynomial (0.0001s)
/////   c   ///// DFT-8             (0.0010s)

MeLOn CheckPoint: SFFT-SUBTRACTION Perform Subtraction takes [0.0021s]
/////   j   ///// Calculate Kab         (0.0019s)
/////   k   ///// Construct DIFF        (0.0002s)

 --||--||--||--||-- EXIT SFFT SUBTRACTION --||--||--||--||-- 

MeLOn Report: SFFT-SUBTRACTION TAKES [0.361 s]
Traceback (most recent call last):
  File "/global/cfs/cdirs/m4385/users/lauren/phrosty/phrosty/pipeline.py", line 559, in <module>
    main()
  File "/global/cfs/cdirs/m4385/users/lauren/phrosty/phrosty/pipeline.py", line 555, in main
    pipeline( args.through_step )
  File "/global/cfs/cdirs/m4385/users/lauren/phrosty/phrosty/pipeline.py", line 465, in __call__
    sfftifier = self.align_and_pre_convolve( templ_image, sci_image )
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/cfs/cdirs/m4385/users/lauren/phrosty/phrosty/pipeline.py", line 180, in align_and_pre_convolve
    with fits.open( sci_image.skysub_path ) as hdul:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/l/laldorot/.conda/envs/diff/lib/python3.11/site-packages/astropy/io/fits/hdu/hdulist.py", line 220, in fitsopen
    raise ValueError(f"Empty filename: {name!r}")
ValueError: Empty filename: None

Should track this down.

laldoroty commented 1 month ago

@rknop can you write a brief summary of the solution and close this?