search

fermi-lat / Likelihood

BSD 3-Clause "New" or "Revised" License
2 stars 1 forks source link

gtlike error: N10optimizers11OutOfBoundsE #85

Closed omersajid9 closed 4 years ago

omersajid9 commented 4 years ago

Hi, I am getting an error while performing the Binned Likelihood analysis of Crab Pulsar. My goal is to generate integrated flux light curves, to analyze Crab flares. I have tried using multiple data sets of Crab and other sources to try to perform the likelihood analysis but am getting the same following error at the end:

Computing TS values for each source (218 total) .......................................Caught N10optimizers11OutOfBoundsE at the top level: Attempt to set the value outside of existing bounds.

I've attached the whole log file of my gtlike command. gtlike error.txt

One more thing to mention is that I was getting a similar error when creating the source XML file. The following is the error: raceback (most recent call last): File "make4FGLxml.py.1", line 824, in if name=='main': cli() File "make4FGLxml.py.1", line 821, in cli sL.makeModel(args.galfile,args.galname,args.isofile,args.isoname,args.normsonly,args.extDir,args.radLim,args.maxRad,args.ExtraRad,args.sigFree,args.varFree,args.psForce,args.E2CAT,args.makeRegion,args.GIndexFree,args.writeDir,args.oldNames) File "make4FGLxml.py.1", line 69, in makeModel addSrcsXML(self,GDfile,GDname,ISOfile,ISOname,oldNames) File "make4FGLxml.py.1", line 155, in addSrcsXML elif float(src.getAttribute('TS_value'))>=sL.sig:

I solved it by editing the make4FGLxml.py file to remove any non-float TS values. As gtlike is an intrinsic command, I dont know how to solve the above problem.

nmirabal commented 4 years ago

Since you are presumably looking at short intervals for Crab flares, the "Caught N10optimizers11OutOfBoundsE at the top level: " message probably originates from free low significance sources that get stuck at the boundaries set in the XML file. You might try freezing low significance sources in the XML file. For that, one could use the -s flag in make4FGLxml.py, the default is 5. You might try experimenting with higher "-s" values until you reach convergence. The REAME file for make4FGLxml.py is at the following link https://fermi.gsfc.nasa.gov/ssc/data/analysis/user/readme_make4FGLxml.txt

omersajid9 commented 4 years ago

@nmirabal Thank you for your response. I am trying out the suggestions but it takes usually a couple of hours to run the command so I'll update after that. There is one another thing that I wanted to ask you. I am getting the following error at the start of gtlike:

Parameter ptsrc not found in parameter group gtlike (at /local/data/glitz1/gssctest/miniconda2/envs/build/conda-bld/fermitools_1580999712705/work/hoops/build/redhat6-x86_64-64bit-gcc48/src/hoops_group.cxx: 67) Using default value of 1 for parameter ptsrc Parameter convol not found in parameter group gtlike (at /local/data/glitz1/gssctest/miniconda2/envs/build/conda-bld/fermitools_1580999712705/work/hoops/build/redhat6-x86_64-64bit-gcc48/src/hoops_group.cxx: 67) Using default value of 1 for parameter convol Parameter resample not found in parameter group gtlike (at /local/data/glitz1/gssctest/miniconda2/envs/build/conda-bld/fermitools_1580999712705/work/hoops/build/redhat6-x86_64-64bit-gcc48/src/hoops_group.cxx: 67) Using default value of 1 for parameter resample Parameter rfactor not found in parameter group gtlike (at /local/data/glitz1/gssctest/miniconda2/envs/build/conda-bld/fermitools_1580999712705/work/hoops/build/redhat6-x86_64-64bit-gcc48/src/hoops_group.cxx: 67) Using default value of 2 for parameter rfactor Parameter minbinsz not found in parameter group gtlike (at /local/data/glitz1/gssctest/miniconda2/envs/build/conda-bld/fermitools_1580999712705/work/hoops/build/redhat6-x86_64-64bit-gcc48/src/hoops_group.cxx: 67) Using default value of 0.1 for parameter minbinsz

Should I be worried about it? I have the latest Fermitools version(1.2.23)

nmirabal commented 4 years ago

Great. Can you please open a separate github issue for the warnings?. That way we can track things better.

omersajid9 commented 4 years ago

@nmirabal Sure. The command ran relatively fast. I had ran it with 3 xml files (-s 50, -s 80 and -s 150) simultaneously. The analysis with the -s 80 xml file completed without any error whereas the other 2 had the same problem. Is there a reason why it didn't run properly for the higher 150 value?

omersajid9 commented 4 years ago

Hi @nmirabal, Is there any way to estimate the range in which we should set the average significance value (-s value), other than using the hit-and-trial method?

nmirabal commented 4 years ago

The Crab nebula is listed at a significance of 27.8 and the Crab pulsar at 39.1, so you have to explore lower cuts. This is an iterative process so it takes some time. The main goal is reaching convergence. But probably a more interactive approach can be taken using python. There are two analysis threads that walk you through the fitting process

https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/python_tutorial.html https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/upper_limits.html

For future inquiries about Fermitools usage rather than code issues please e-mail the helpdesk directly at fermihelp@milkyway.gsfc.nasa.gov

omersajid9 commented 4 years ago

Thank you