search

fermi-lat / Likelihood

BSD 3-Clause "New" or "Revised" License
2 stars 1 forks source link

gtsrcmaps not working properly #36

Closed migaraya closed 3 years ago

migaraya commented 5 years ago

I am running the new fermi software in a Mac OS, Mojave 10.14.3. When I run 'conda list fermi' I get the following info

fermitools 1.0.1 py27h5b5c8cd_0 fermi

fermitools-data 0.16 0 fermi

for standard P8R3_SOURCE_V2 analysis.

I've had issues running gtsrcmaps (the convolution of the model and PSF). The xml model file has the standard form, made with the make4FGLxml.py script. I have followed the usual steps in the binned analysis

https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/binned_likelihood_tutorial.html

It seems that the tool 'gtsrcmaps' simply does not calculate the correct source map file. It outputs some files with no error, but the tools don't seem to read any of the files later on. The files (and their sizes) created by gtsrcmaps are the following:

-rw-r--r-- 1 fermi staff 10558080 Mar 23 19:25 binned_srcmaps.fits

-rw-r--r-- 1 fermi staff 5122080000 Mar 23 19:17 binned_srcmaps.fits_0.fits

-rw-r--r-- 1 fermi staff 2233226880 Mar 23 19:25 binned_srcmaps.fits_1.fits

(I forgot to check what each of these files contained and deleted them, but I could make them again if necessary)

In the next steps when I run gtlike and gtmodel the source maps are calculated again in the process for all sources in the xml file. Every time there is a fit, the source maps are calculated again as if the file(s) produced by gtsrcmaps was never used.

jasercion commented 5 years ago

Just to clarify, are the files generated by gtsrcmaps themselves correct? Is the problem just that the tools aren't using the generated maps further down the analysis chain or is there also an issue with the gtsrcmaps output files (i.e. file corruption, incorrect calculation, etc)?

migaraya commented 5 years ago

Yeah the files are generated by gtsrcmaps, there is no output error at that point. The tools that follow, such as gtlike, do not use any of those files, I assume, because during the fit the terminal shows the calculation of the source maps for all the sources again in the process. I forgot to check the files to see if they were corrupted or what each contained. I could make them again to see.

migaraya commented 5 years ago

I mean the analysis can be completed, it only takes a lot of time because the source maps are generated during every fit.

migaraya commented 5 years ago

An update: the problem described above does not happen when the number of sources included in the model is "reasonable". I was doing a test where I had a 40-deg radius ROI and about 700 sources. I deleted half of them just to try something out and did not see the problem.

migaraya commented 4 years ago

El 02-03-2020 17:29, jasercion escribió:

@migaraya [1] Are you still running into this problem with the latest version of the tools?

-- You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub [2], or unsubscribe [3].

hello,

yes but only for very large ROIs (>~ 40 deg), ie, not for typical analyses. However, I think it is an issue related to the lack of space in my computer, so probably not an issue with the tools.

-- Miguel Araya, Escuela de Física, oficina 408, casillero 67. Universidad de Costa Rica, San Jose, Costa Rica

Links:

[1] https://github.com/migaraya [2] https://github.com/fermi-lat/Likelihood/issues/36?email_source=notifications&email_token=ALV35PBEV5SU6V7JRH76YALRFQ6LZA5CNFSM4HB26GSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENRMS4Q#issuecomment-593676658 [3] https://github.com/notifications/unsubscribe-auth/ALV35PAMDQTZY6TXTTMY6UTRFQ6LZANCNFSM4HB26GSA