search

gammapy / enrico

Enrico helps you with your Fermi data analysis
http://enrico.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
22 stars 26 forks source link

Problems with enrico_lc #159

Closed ErickR100 closed 3 years ago

ErickR100 commented 3 years ago

Hi

I try to generate the Mrk421 light curve with time intervals in a text file, but when executing enrico_lc with the configuration file, I get the following error:

['/home/erick/enrico/bin/enrico_lc', 'mrk421.conf'] [Message]: work on the config file mrk421.conf {'out': '/home/erick/Escritorio/Mkn421_intt/datos', 'verbose': 'yes', 'clobber': 'yes', 'Submit': 'no', 'target': {'name': 'Mrk421', 'ra': 166.113808, 'dec': 38.20883287, 'redshift': 0.031, 'ebl_model': 4, 'spectrum': 'PowerLaw', 'fit_tau': 'no'}, 'space': {'xref': 166.113808, 'yref': 38.20883287, 'rad': 15.0, 'binsz': 0.1, 'coordsys': 'CEL', 'proj': 'AIT', 'phibins': 0}, 'file': {'spacecraft': 'sc.fits', 'event': 'fotones.dat', 'xml': '/home/erick/Escritorio/Mkn421_intt/datos/Mrk421_PowerLaw_model.xml', 'tag': 'LAT_Analysis', 'parent_config': ''}, 'time': {'tmin': 0.0, 'tmax': 0.0, 'file': '/home/erick/Escritorio/Mkn421_intt/tiempo.dat', 'type': 'MET'}, 'energy': {'emin': 100.0, 'emax': 300000.0, 'enumbins_per_decade': 10, 'decorrelation_energy': 'no'}, 'event': {'irfs': 'CALDB', 'evclass': 128, 'evtype': 3}, 'analysis': {'zmax': 90.0, 'likelihood': 'binned', 'EnergyDispersion': 'yes', 'ComputeDiffrsp': 'yes', 'roicut': 'no', 'evtroicuts': 'yes', 'evttimecuts': 'yes', 'filter': '(DATA_QUAL>0)&&(LAT_CONFIG==1)', 'keep_all_srcmaps': 'no'}, 'environ': {'FERMI_DATA_DIR': '', 'FERMI_CATALOG_DIR': '', 'FERMI_CATALOG': '', 'FERMI_DIFFUSE_DIR': '', 'FERMI_PREPROCESSED_DIR': ''}, 'fitting': {'optimizer': 'MINUIT', 'ftol': 1e-06, 'min_source_TS': 1.0}, 'model': {'diffuse_gal_dir': '', 'diffuse_iso_dir': '', 'diffuse_gal': '', 'diffuse_iso': '', 'min_significance_catalog': 4.0, 'min_significance_free': 25.0, 'max_radius': 5.0, 'max_roi': 10.0, 'parameters_noise': 0.0}, 'Spectrum': {'FitsGeneration': 'yes', 'ResultPlots': 'yes', 'ResultParentPlots': 'no', 'FrozenSpectralIndex': 0.0}, 'UpperLimit': {'SpectralIndex': 2.0, 'Method': 'Profile', 'envelope': 'no', 'TSlimit': 25.0, 'cl': 0.95}, 'LightCurve': {'FitsGeneration': 'yes', 'NLCbin': 1, 'SpectralIndex': 2.0, 'MakeConfFile': 'yes', 'BayesianBlocks': 'no', 'ComputeVarIndex': 'yes', 'TSLightCurve': 9.0, 'DiagnosticPlots': 'yes', 'DiagnosticParentPlots': 'no'}, 'BayesianBlocks': {'FitsGeneration': 'yes', 'p0': 0.01, 'SpectralIndex': 1.8, 'MakeConfFile': 'yes', 'TSLightCurve': 9.0, 'DiagnosticPlots': 'yes', 'DiagnosticParentPlots': 'no'}, 'FoldedLC': {'NLCbin': 10, 'epoch': 0.0, 'Period': 0.0}, 'AppLC': {'FitsGeneration': 'yes', 'index': 2.0, 'rad': 1.0, 'NLCbin': 10, 'binsFromData': 'no', 'emin': -1.0, 'emax': -1.0}, 'Ebin': {'FitsGeneration': 'yes', 'NumEnergyBins': 0, 'TSEnergyBins': 9.0, 'DistEbins': 'logE'}, 'TSMap': {'Re-Fit': 'no', 'npix': 10, 'RemoveTarget': 'yes', 'method': 'row'}, 'findsrc': {'FitsGeneration': 'yes', 'Refit': 'yes'}, 'srcprob': {'FitsGeneration': 'yes', 'rad': 1.0, 'srclist': '', 'numberPhoton': 10}, 'Contours': {'parname1': 'Prefactor', 'parname2': 'Index'}, 'ComponentAnalysis': {'EUnBinned': -1.0, 'FrontBack': 'no', 'PSF': 'no', 'EDISP': 'no', 'FGL4': 'no'}} reusing /home/erick/Escritorio/Mkn421_intt/datos/Mrk421_LAT_Analysis_EvtCoarse.fits as event file to speed-up the analysis use /home/erick/Escritorio/Mkn421_intt/tiempo.dat [LightCurve]: Running LC with 9.0 bins

Traceback (most recent call last): File "/home/erick/enrico/bin/enrico_lc", line 26, in liste = np.genfromtxt(sys.argv[1],dtype="str",unpack=True) File "/home/erick/anaconda3/envs/fermi/lib/python3.7/site-packages/numpy/lib/npyio.py", line 2122, in genfromtxt raise ValueError(errmsg) ValueError: Some errors were detected ! Line #9 (got 1 columns instead of 3) Line #20 (got 1 columns instead of 3) Line #29 (got 1 columns instead of 3) Line #37 (got 1 columns instead of 3) Line #43 (got 1 columns instead of 3) Line #49 (got 1 columns instead of 3) Line #54 (got 1 columns instead of 3) Line #70 (got 1 columns instead of 3) Line #79 (got 1 columns instead of 3) Line #85 (got 1 columns instead of 3) Line #101 (got 1 columns instead of 3) Line #111 (got 1 columns instead of 3) Line #123 (got 1 columns instead of 3) Line #142 (got 1 columns instead of 3) Line #158 (got 1 columns instead of 3) Line #166 (got 1 columns instead of 3) Line #181 (got 1 columns instead of 3) Line #195 (got 1 columns instead of 3) Line #206 (got 1 columns instead of 3) Line #212 (got 1 columns instead of 3) Line #222 (got 1 columns instead of 3) Line #226 (got 1 columns instead of 3)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/erick/enrico/bin/enrico_lc", line 32, in run_over_conffile(sys.argv[1]) File "/home/erick/enrico/bin/enrico_lc", line 20, in run_over_conffile lc.MakeLC() File "/home/erick/enrico/enrico/lightcurve.py", line 225, in MakeLC self._MakeTimeBins() File "/home/erick/enrico/enrico/lightcurve.py", line 119, in _MakeTimeBins for i in range(self.Nbin): TypeError: 'float' object cannot be interpreted as an integer

##################################################################################################################################################################################################################################################################################################################

I have the following in my config file:

[time] tmin = 441763203 tmax = 454809603 file = "/home/erick/Escritorio/Mkn421_intt/tiempo.dat" type = MET

[LightCurve]

Generates fits files or not?

    FitsGeneration = yes
    #Number of points for the LC
    NLCbin = 1
    #Index for the power law. Left free to vary if 0
    SpectralIndex = 2.0
    MakeConfFile = yes
    #Bayesian blocks
    BayesianBlocks = no
    #Compute Variability index as in the 2FGL.
    ComputeVarIndex = yes
    #Compute an UL if the TS of the sources is <TSLightCurve
    TSLightCurve = 9.0
    #Generates control plots
    DiagnosticPlots = yes
    # Plot (update) the parent config related plots during time bin production
    DiagnosticParentPlots = no
davidsanchez commented 3 years ago

Hi how your /tiempo.dat looks like? best

ErickR100 commented 3 years ago

It is as follows ############################################################################################################################################################################################################

441763203 446860803 446860804 449539203 449539205 450748803 450748806 451353603 451353607 451699203 451699208 451785603 451785609 451807203 451807200 451828803 451828801 454420803

davidsanchez commented 3 years ago

With the # ? not sur this is well read.

ErickR100 commented 3 years ago

Without the # signs, only the 2 columns with the first the start time and the second the end time

davidsanchez commented 3 years ago

ok. I'll have a deeper look asap

ErickR100 commented 3 years ago

Thank you so much

davidsanchez commented 3 years ago

Could you give me the full config file please?

davidsanchez commented 3 years ago

Could you try the fix I made ?thanks

ErickR100 commented 3 years ago

Default config and validation file for Enrico.

#

Besides providing default options it is also used

to check the user config file.

Options without default options here must be

specified in the user config file

Folder where the output will be stored

out = /home/erick/Escritorio/Mkn421_intt/datos

verbosity

verbose = yes

Overwrite existing output files. Use with caution

clobber = yes

Submit the job to a cluster?

Submit = no

[target] name = Mrk421 ra = 166.113808 dec = 38.20883287 redshift = 0.031 ebl_model = 4 spectrum = PowerLaw

0=Kneiske, 1=Primack05, 2=Kneiske_HighUV, 3=Stecker05, 4=Franceschini, 5=Finke, 6=Gilmore, 10=Dominguez11

# full list http://www-glast.stanford.edu/cgi-bin/viewcvs/celestialSources/eblAtten/eblAtten/EblAtten.h?view=markup
fit_tau = no

[space] xref = 166.113808 yref = 38.20883287 rad = 15.0 binsz = 0.1 coordsys = CEL proj = AIT phibins = 0

[file] spacecraft = sc.fits event = fotones.dat xml = /home/erick/Escritorio/Mkn421_intt/datos/Mrk421_PowerLaw_model.xml tag = LAT_Analysis

keep track of the parent config during nested operations (e.g. energy/time bins)

parent_config = ""

[time] tmin = 441763203.0 tmax = 454809603.0 file = "/home/erick/Escritorio/Mkn421_intt/tiempo.txt" type ='MET'

[energy] emin = 100.0 emax = 300000.0 enumbins_per_decade = 10 decorrelation_energy = no

[event] irfs = CALDB evclass = 128 evtype = 3

[analysis] zmax = 100

General analysis options

likelihood = unbinned
# Correct energy dispersion?
EnergyDispersion = yes
ComputeDiffrsp = yes
roicut = no
# cuts applied during the first coarse bin selection:
evtroicuts = yes
evttimecuts = yes
filter = (DATA_QUAL>0)&&(LAT_CONFIG==1)
# should we keep the srcmaps for all sources in the srcMap file?
# this should speed up future fits, at the cost of disk space.
keep_all_srcmaps = no

[environ]

Analysis environment configuration

# Can also be done via shell environment variables
FERMI_DATA_DIR = ""
FERMI_CATALOG_DIR = ""
FERMI_CATALOG = ""
FERMI_DIFFUSE_DIR = ""
FERMI_PREPROCESSED_DIR = ""

[fitting] optimizer = MINUIT ftol = 1e-06

if source_ts < min_source_TS in the fit, the source is removed from the model

min_source_TS = 1.0

[model]

The following options determine the xml model

diffuse_gal_dir = ""
diffuse_iso_dir = ""
diffuse_gal = ""
diffuse_iso = ""
# user points sources for diffuse catalog sources
# freeze spectral parameters for weak and far away sources:
min_significance_catalog = 4.0
min_significance_free = 25.0
max_radius = 5.0
# Add all source in a radius of analyse ROI + this radius
max_roi = 10.0
# Add gaussian noise to the spectral parameters of free sources (may help during the fit)
parameters_noise = 0.0

[Spectrum]

Generates fits files or not?

FitsGeneration = yes
#Generates plots (SED, model map)
ResultPlots = yes
# Plot (update) the parent config related plots during energy bin production
ResultParentPlots = no
#Freeze the spectral index of the source. Has no implication if 0 (Left free to vary)
FrozenSpectralIndex = 0.0

[UpperLimit]

Assumed Spectral index

SpectralIndex = 2.0
# UL method could be Profile or Integral (provided by the fermi collaboration).
#Poisson is base on the Feldman-Cousins method for low signal. Only 95 % yet available
Method = Profile
envelope = no
#Compute an UL if the TS of the sources is <TSlimit
TSlimit = 25.0
# Confidence level for the Ul computation
cl = 0.95

[LightCurve]

Generates fits files or not?

FitsGeneration = yes
#Number of points for the LC
NLCbin = 1
#Index for the power law. Left free to vary if 0
SpectralIndex = 2.0
MakeConfFile = yes
#Bayesian blocks
BayesianBlocks = no
#Compute Variability index as in the 2FGL.
ComputeVarIndex = yes
#Compute an UL if the TS of the sources is <TSLightCurve
TSLightCurve = 9.0
#Generates control plots
DiagnosticPlots = yes
# Plot (update) the parent config related plots during time bin production
DiagnosticParentPlots = no

[BayesianBlocks]

Generates fits files or not?

FitsGeneration = yes
#False alarm probability
p0 = 0.01
#Index for the power law. Left free to vary if 0
SpectralIndex = 1.8
MakeConfFile = yes
#Compute an UL if the TS of the sources is <TSLightCurve
TSLightCurve = 9.0
#Generates control plots
DiagnosticPlots = yes
# Plot (update) the parent config related plots during time bin production
DiagnosticParentPlots = no

[FoldedLC]

Number of bins for the orbitally folded LC

NLCbin = 10
#Epoch of phase=0 in MJD, equal to tmin is 0
epoch = 0.0
#Orbital period in days
Period = 0.0

[AppLC]

Generates fits files or not?

FitsGeneration = yes
#Spectral index for the exposure calculation
index = 2.0
#Apperture radius
rad = 1.0
#Number of bins
NLCbin = 10
#bin form data or frozen bin size
binsFromData = no
#minimum energy for the apperture lightcurve, if -1 use the one define in energy section
emin = -1.0
emax = -1.0

[Ebin]

Generates fits files or not?

FitsGeneration = yes
NumEnergyBins = 0
#Compute an UL if the TS of the sources is <TSEnergyBins
TSEnergyBins = 9.0
# How to generate the Ebins
# - logE: log scaled bins
# - TS: similar TS per bin
# - mix: half-way between log and TS.
# - [ E1, E2, E3, E4, ..., EN ]: custom binning. 
# - default is logE
DistEbins = logE

[TSMap]

Re-fit before computing the TS map

Re-Fit = no
#Numbers of pixel in x and y
npix = 10
#Remove or not the target from the model
RemoveTarget = yes
#Generate the TS map pixel by pixel or by grouping the pixels by row.
#(reduce the numbers of jobs but each job is longer)
method = row

[findsrc]

Generates fits files or not?

FitsGeneration = yes
#Reoptimize before
Refit = yes

[srcprob]

Generates fits files or not?

FitsGeneration = yes
#radius for the computation
rad = 1.0
# list of sources
srclist = ""
# number of photons to print
numberPhoton = 10

[Contours] parname1 = Prefactor parname2 = Index

[ComponentAnalysis]

EUnBinned to the energy (MeV) where

# binned and unbinned analysis should be split
# disabled if -1
EUnBinned = -1.0
# Divide the analysis in FrontBack classes
FrontBack = no
# Divide the analysis in PSF classes
PSF = no
# Divide the analysis in EDISP classes
EDISP = no
# Divide the analysis in EDISP classes
FGL4 = no
ErickR100 commented 3 years ago

Could you try the fix I made ?thanks Sorry, but what solution?

davidsanchez commented 3 years ago

I made a push . could you update your enrico?

Sahakyan86 commented 3 years ago

Hi,

I had a similar issue but the new version is fine.

davidsanchez commented 3 years ago

great.

ErickR100 commented 3 years ago

thank you so much, everything works very well