search

gbrammer / grizli

Grizli: The "Grism redshift and line" analysis software
MIT License
65 stars 50 forks source link

RuntimeError: Problem during: drizCR._driz_cr(), exitcode: 1. Check log. #185

Open DazhiDTU opened 7 months ago

DazhiDTU commented 7 months ago

I am trying to reduce NIRCam imaging from the CEERS notebook. However, the pipeline always fails to run the drizCR. It seems like grizli attempts to overwrite some file but never success...

Many thanks!

2023-12-07 23:03:29,910 - stpipe - ERROR - Traceback (most recent call last): Traceback (most recent call last): 2023-12-07 23:03:29,912 - stpipe - ERROR - File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap 2023-12-07 23:03:29,914 - stpipe - ERROR - self.run() self.run() 2023-12-07 23:03:29,916 - stpipe - ERROR - File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/multiprocessing/process.py", line 108, in run File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/multiprocessing/process.py", line 108, in run 2023-12-07 23:03:29,917 - stpipe - ERROR - self._target(*self._args, *self._kwargs) self._target(self._args, **self._kwargs) 2023-12-07 23:03:29,920 - stpipe - ERROR - File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/drizzlepac/drizCR.py", line 294, in _driz_cr File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/drizzlepac/drizCR.py", line 294, in _driz_cr 2023-12-07 23:03:29,923 - stpipe - ERROR - util.createFile(cr_mask.astype(np.uint8), util.createFile(cr_mask.astype(np.uint8), 2023-12-07 23:03:29,926 - stpipe - ERROR - File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/drizzlepac/util.py", line 1153, in createFile File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/drizzlepac/util.py", line 1153, in createFile 2023-12-07 23:03:29,928 - stpipe - ERROR - fitsobj.writeto(outfile) fitsobj.writeto(outfile) 2023-12-07 23:03:29,930 - stpipe - ERROR - File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/astropy/io/fits/hdu/hdulist.py", line 1010, in writeto File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/astropy/io/fits/hdu/hdulist.py", line 1010, in writeto 2023-12-07 23:03:29,931 - stpipe - ERROR - fileobj = _File(fileobj, mode=mode, overwrite=overwrite) fileobj = _File(fileobj, mode=mode, overwrite=overwrite) 2023-12-07 23:03:29,936 - stpipe - ERROR - File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/astropy/io/fits/file.py", line 217, in init File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/astropy/io/fits/file.py", line 217, in init 2023-12-07 23:03:29,938 - stpipe - ERROR - self._open_filename(fileobj, mode, overwrite) self._open_filename(fileobj, mode, overwrite) 2023-12-07 23:03:29,940 - stpipe - ERROR - File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/astropy/io/fits/file.py", line 615, in _open_filename File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/astropy/io/fits/file.py", line 615, in _open_filename 2023-12-07 23:03:29,943 - stpipe - ERROR - self._overwrite_existing(overwrite, None, True) self._overwrite_existing(overwrite, None, True) 2023-12-07 23:03:29,946 - stpipe - ERROR - File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/astropy/io/fits/file.py", line 498, in _overwrite_existing File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/astropy/io/fits/file.py", line 498, in _overwrite_existing 2023-12-07 23:03:29,951 - stpipe - ERROR - raise OSError(NOT_OVERWRITING_MSG.format(self.name)) raise OSError(NOT_OVERWRITING_MSG.format(self.name)) 2023-12-07 23:03:29,953 - stpipe - ERROR - OSError: File jw01791003001_03101_00003_nrcb3_sci1_crmask.fits already exists. If you mean to replace it then use the argument "overwrite=True". OSError: File jw01791003001_03101_00003_nrcb3_sci1_crmask.fits already exists. If you mean to replace it then use the argument "overwrite=True". 2023-12-07 23:03:32,184 - stpipe - ERROR - ############################################################################## ############################################################################## 2023-12-07 23:03:32,188 - stpipe - ERROR - # #

2023-12-07 23:03:32,190 - stpipe - ERROR - # ERROR: #

ERROR:

2023-12-07 23:03:32,191 - stpipe - ERROR - # AstroDrizzle Version 3.6.1 encountered a problem! Processing terminated #

AstroDrizzle Version 3.6.1 encountered a problem! Processing terminated

2023-12-07 23:03:32,192 - stpipe - ERROR - # at 23:03:32.181 (07/12/2023). #

at 23:03:32.181 (07/12/2023).

2023-12-07 23:03:32,194 - stpipe - ERROR - # #

2023-12-07 23:03:32,195 - stpipe - ERROR - ############################################################################## ############################################################################## 2023-12-07 23:03:32,279 - stpipe - ERROR - An unhandled exception ocurred: Traceback (most recent call last): File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/drizzlepac/util.py", line 230, in wrapper result = func(*args, **kwargs) File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/drizzlepac/astrodrizzle.py", line 230, in run drizCR.rundrizCR(imgObjList, configobj, procSteps=procSteps) File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/drizzlepac/drizCR.py", line 92, in rundrizCR mputil.launch_and_wait(subprocs, pool_size) # blocks till all done File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/stsci/tools/mputil.py", line 103, in launch_and_wait raise RuntimeError("Problem during: "+str(p.process.name)+ \ RuntimeError: Problem during: drizCR._driz_cr(), exitcode: 1. Check log.

jdavies-st commented 4 months ago

You are trying to overwrite an existing file jw01791003001_03101_00003_nrcb3_sci1_crmask.fits, and Drizzlepac doesn't like that. Make sure the file doesn't already exist.

2023-12-07 23:03:29,910 - stpipe - ERROR - Traceback (most recent call last):
Traceback (most recent call last):
2023-12-07 23:03:29,912 - stpipe - ERROR - File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2023-12-07 23:03:29,914 - stpipe - ERROR - self.run()
self.run()
2023-12-07 23:03:29,916 - stpipe - ERROR - File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/multiprocessing/process.py", line 108, in run
File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/multiprocessing/process.py", line 108, in run
2023-12-07 23:03:29,917 - stpipe - ERROR - self._target(*self._args, **self._kwargs)
self._target(*self._args, **self._kwargs)
2023-12-07 23:03:29,920 - stpipe - ERROR - File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/drizzlepac/drizCR.py", line 294, in _driz_cr
File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/drizzlepac/drizCR.py", line 294, in _driz_cr
2023-12-07 23:03:29,923 - stpipe - ERROR - util.createFile(cr_mask.astype(np.uint8),
util.createFile(cr_mask.astype(np.uint8),
2023-12-07 23:03:29,926 - stpipe - ERROR - File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/drizzlepac/util.py", line 1153, in createFile
File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/drizzlepac/util.py", line 1153, in createFile
2023-12-07 23:03:29,928 - stpipe - ERROR - fitsobj.writeto(outfile)
fitsobj.writeto(outfile)
2023-12-07 23:03:29,930 - stpipe - ERROR - File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/astropy/io/fits/hdu/hdulist.py", line 1010, in writeto
File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/astropy/io/fits/hdu/hdulist.py", line 1010, in writeto
2023-12-07 23:03:29,931 - stpipe - ERROR - fileobj = _File(fileobj, mode=mode, overwrite=overwrite)
fileobj = _File(fileobj, mode=mode, overwrite=overwrite)
2023-12-07 23:03:29,936 - stpipe - ERROR - File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/astropy/io/fits/file.py", line 217, in init
File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/astropy/io/fits/file.py", line 217, in init
2023-12-07 23:03:29,938 - stpipe - ERROR - self._open_filename(fileobj, mode, overwrite)
self._open_filename(fileobj, mode, overwrite)
2023-12-07 23:03:29,940 - stpipe - ERROR - File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/astropy/io/fits/file.py", line 615, in _open_filename
File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/astropy/io/fits/file.py", line 615, in _open_filename
2023-12-07 23:03:29,943 - stpipe - ERROR - self._overwrite_existing(overwrite, None, True)
self._overwrite_existing(overwrite, None, True)
2023-12-07 23:03:29,946 - stpipe - ERROR - File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/astropy/io/fits/file.py", line 498, in _overwrite_existing
File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/astropy/io/fits/file.py", line 498, in _overwrite_existing
2023-12-07 23:03:29,951 - stpipe - ERROR - raise OSError(NOT_OVERWRITING_MSG.format(self.name))
raise OSError(NOT_OVERWRITING_MSG.format(self.name))
2023-12-07 23:03:29,953 - stpipe - ERROR - OSError: File jw01791003001_03101_00003_nrcb3_sci1_crmask.fits already exists. If you mean to replace it then use the argument "overwrite=True".
OSError: File jw01791003001_03101_00003_nrcb3_sci1_crmask.fits already exists. If you mean to replace it then use the argument "overwrite=True".
2023-12-07 23:03:32,184 - stpipe - ERROR - ##############################################################################
##############################################################################
2023-12-07 23:03:32,188 - stpipe - ERROR - # #

2023-12-07 23:03:32,190 - stpipe - ERROR - # ERROR: #

ERROR:

2023-12-07 23:03:32,191 - stpipe - ERROR - # AstroDrizzle Version 3.6.1 encountered a problem! Processing terminated #

AstroDrizzle Version 3.6.1 encountered a problem! Processing terminated

2023-12-07 23:03:32,192 - stpipe - ERROR - # at 23:03:32.181 (07/12/2023). #

at 23:03:32.181 (07/12/2023).

2023-12-07 23:03:32,194 - stpipe - ERROR - # #

2023-12-07 23:03:32,195 - stpipe - ERROR - ##############################################################################
##############################################################################
2023-12-07 23:03:32,279 - stpipe - ERROR - An unhandled exception ocurred:
Traceback (most recent call last):
File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/drizzlepac/util.py", line 230, in wrapper
result = func(*args, **kwargs)
File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/drizzlepac/astrodrizzle.py", line 230, in run
drizCR.rundrizCR(imgObjList, configobj, procSteps=procSteps)
File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/drizzlepac/drizCR.py", line 92, in rundrizCR
mputil.launch_and_wait(subprocs, pool_size) # blocks till all done
File "/Users/dzhou/anaconda3/envs/grizlihst39/lib/python3.9/site-packages/stsci/tools/mputil.py", line 103, in launch_and_wait
raise RuntimeError("Problem during: "+str(p.process.name)+
RuntimeError: Problem during: drizCR._driz_cr(), exitcode: 1. Check log.