Open Pablogh292 opened 4 months ago
Doing some further digging it would seem that the problem is with error exception inside lightcurve.py because when gtanalysis.py raises a RuntimeError, lightcurve.py somehow doesn't catch it and just crashes because of the Error. Also tried with the newest version of fermipy and the issue persists.
I posted a comment in your PR
Hi, I am getting similar issue while making lightcurve of a fermi source [7day binning]. How do I account for this? I have attached the error, code and config files.
2024-05-24 06:03:34 INFO lightcurve_542505601_543110401 GTBinnedAnalysis.write_xml(): Writing /home/navaneeth/Works/mrk421/mrk421_LC_batch/outdir_lc/lightcurve_542505601_543110401/base_00.xml...
Joint fit ['4FGL J1104.4+3812', '4FGL J1131.0+3815', '4FGL J1112.5+3448', '4FGL J1127.8+3618', '4FGL J1106.7+3623']
Fitting shape 4FGL J1104.4+3812 TS: 142.949
2024-05-24 06:03:35 INFO lightcurve_542505601_543110401 GTBinnedAnalysis.write_xml(): Writing /home/navaneeth/Works/mrk421/mrk421_LC_batch/outdir_lc/lightcurve_542505601_543110401/fit_model_final_00.xml...
2024-05-24 06:03:36 INFO lightcurve_542505601_543110401 GTAnalysis._process_lc_bin(): Finished time range 542505601 543110401
2024-05-24 06:03:38 INFO lightcurve_543110401_543715201 GTAnalysis._process_lc_bin(): Fitting time range 543110401 543715201
2024-05-24 06:03:39 INFO lightcurve_543110401_543715201 GTBinnedAnalysis._create_ltcube(): Generating local LT cube.
Analysis failed in time range 543110401 543715201
(<class 'RuntimeError'>, RuntimeError('irfs.py(569): Zero exposure!'), <traceback object at 0x7fbd72e36780>)
2024-05-24 06:03:41 ERROR GTAnalysis._make_lc(): Fit failed in bin 498 in range 543110401 543715201.
2024-05-24 06:03:43 INFO lightcurve_543715201_544320001 GTAnalysis._process_lc_bin(): Fitting time range 543715201 544320001
2024-05-24 06:03:44 INFO lightcurve_543715201_544320001 GTBinnedAnalysis._create_ltcube(): Generating local LT cube.
Analysis failed in time range 543715201 544320001
(<class 'RuntimeError'>, RuntimeError('irfs.py(569): Zero exposure!'), <traceback object at 0x7fbd7269efc0>)
2024-05-24 06:03:45 ERROR GTAnalysis._make_lc(): Fit failed in bin 499 in range 543715201 544320001.
2024-05-24 06:03:47 INFO lightcurve_544320001_544924801 GTAnalysis._process_lc_bin(): Fitting time range 544320001 544924801
2024-05-24 06:03:48 INFO lightcurve_544320001_544924801 GTBinnedAnalysis._create_ltcube(): Generating local LT cube.
WARNING: FITSFixedWarning: RADECSYS= 'FK5 '
the RADECSYS keyword is deprecated, use RADESYSa. [astropy.wcs.wcs]
2024-05-24 06:06:01 INFO lightcurve_544320001_544924801 GTBinnedAnalysis.write_xml(): Writing /home/navaneeth/Works/mrk421/mrk421_LC_batch/outdir_lc/lightcurve_544320001_544924801/base_00.xml...
Joint fit ['4FGL J1104.4+3812']
2024-05-24 06:06:01 WARNING lightcurve_544320001_544924801 GTAnalysis.profile_norm(): Source model for source 4FGL J1051.4+3942 has no counts in ROI
2024-05-24 06:06:01 WARNING lightcurve_544320001_544924801 GTAnalysis.profile_norm(): Upper limit not found. Refitting normalization.
Traceback (most recent call last):
File "/home/navaneeth/Works/mrk421/mrk421_LC_batch/run_lc.py", line 9, in <module>
lc = gta.lightcurve('4FGL J1104.4+3812', binsz=86400.*7.0, free_radius=3.0, prefix='Mrk421_FullLC', outdir='outdir_lc')
File "/home/navaneeth/miniconda3/envs/fermipy/lib/python3.9/site-packages/fermipy/lightcurve.py", line 290, in lightcurve
o = self._make_lc(name, **config)
File "/home/navaneeth/miniconda3/envs/fermipy/lib/python3.9/site-packages/fermipy/lightcurve.py", line 451, in _make_lc
next_fit = next(mapo)
File "/home/navaneeth/miniconda3/envs/fermipy/lib/python3.9/site-packages/fermipy/lightcurve.py", line 181, in _process_lc_bin
fit_results = _fit_lc(gta, name, **kwargs)
File "/home/navaneeth/miniconda3/envs/fermipy/lib/python3.9/site-packages/fermipy/lightcurve.py", line 59, in _fit_lc
gta.fit()
File "/home/navaneeth/miniconda3/envs/fermipy/lib/python3.9/site-packages/fermipy/gtanalysis.py", line 3074, in fit
self.update_source(name, reoptimize=config['reoptimize'])
File "/home/navaneeth/miniconda3/envs/fermipy/lib/python3.9/site-packages/fermipy/gtanalysis.py", line 4106, in update_source
sd = self.get_src_model(name, paramsonly, reoptimize, npts,
File "/home/navaneeth/miniconda3/envs/fermipy/lib/python3.9/site-packages/fermipy/gtanalysis.py", line 4264, in get_src_model
lnlp = self.profile_norm(name, savestate=True,
File "/home/navaneeth/miniconda3/envs/fermipy/lib/python3.9/site-packages/fermipy/gtanalysis.py", line 2472, in profile_norm
o = self.profile(name, parName,
File "/home/navaneeth/miniconda3/envs/fermipy/lib/python3.9/site-packages/fermipy/gtanalysis.py", line 2684, in profile
raise RuntimeError(
RuntimeError: Parameter scan points for 4FGL J1051.4+3942::norm include infinite value.
import numpy as np
from fermipy.gtanalysis import GTAnalysis
gta = GTAnalysis('config.yaml', logging={'verbosity': 3})
gta.setup()
gta.optimize()
lc = gta.lightcurve('4FGL J1104.4+3812', binsz=86400.*7.0, free_radius=3.0, prefix='Mrk421_FullLC', outdir='outdir_lc')
data:
evfile : pfiles.dat
scfile : sfiles.dat
binning:
roiwidth : 10.0
binsz : 0.1
binsperdec : 8
selection :
emin : 100
emax : 500000
zmax : 90
evclass : 128
evtype : 3
tmin : 241920001
tmax : 730771205
target : '4FGL J1104.4+3812'
gtlike:
edisp : True
irfs : 'P8R3_SOURCE_V3'
edisp_disable : ['isodiff','galdiff']
model:
src_roiwidth : 15.0
galdiff : '/home/navaneeth/Works/MY_FERMI_DIR/gll_iem_v07.fits'
isodiff : '/home/navaneeth/Works/MY_FERMI_DIR/iso_P8R3_SOURCE_V3_v1.txt'
catalogs : '/home/navaneeth/Works/MY_FERMI_DIR/gll_psc_v34.fit'
Describe the error When using fermipy to generate lightcurves of Mrk 501 with daily bins I noticed that sometimes the code would fail when it would encounter a bin with "infinite values", this would cause the entire code to crash and not generate the lightcurve fits file.
To Reproduce
Expected behavior It was my understanding that the code should be able to simply ignore this bin and continue onwards to the next, but for some reason when it calls for gtanalyisis.py, if it finds NaN values it raises an error and crashes the entire code instead.
Log files
Environment (please complete the following information):
Additional context It should be noted that this issue happens even in the most recent version of fermipy.