About filter expressions in the [analysis] field

[pmunar]

Hi everyone,

There is a problem with the filter expressions that can be set into the [analysis] section. I am trying to make a more precise filtering of the data adding &&(ANGSEP(RA_src,DEC_src,RA_SCZ,DEC_SCZ) <50 ) to the filter, and it fails verification. I could not find in the code where is the verification part. I tried this same expression with the normal Fermi Science tools and it works.

Thanks.

Pere.

[davidsanchez]

Hi, I also try and it is not working.I am not sure that ANGSEP is defined or recognized. a workaround can be to use gtselect with a ROI of 50degrees cheers david

[pmunar]

Within the Sciente Tools ANGSEP is included. What I think is that within Enrico it is not recognized. The solution that you suggest is not what I want. I need to select time intervals in which my source of interest is at an angular distance from the center of the Fermi FOV lower than 50 deg. In the ROI selection you choose how wide your FOV will be, but there might be events recorded large off-axis angles, which are not interesting for what I want to do. Thanks anyway for your reply. Cheers, Pere.

[zblz]

This seems to be a bug in the config validator. The configspec only says:

    filter = string(default='(DATA_QUAL>0)&&(LAT_CONFIG==1)')

so any string should be valid. When tested manually with the is_string function, the complete string validates fine:

>>> from enrico.extern.validate import Validator, is_string
>>> value = 'DATA_QUAL==1&&LAT_CONFIG==1&&ABS(ROCK_ANGLE)<52&&(ANGSEP(RA_src,DEC_src,RA_SCZ,DEC_SCZ)<50)'
>>> is_string(value) == value
True

However, the full config validation with the above value set as [analysis]/[filter] fails:

>>> from enrico.extern.configobj import ConfigObj, flatten_errors
>>> from enrico.extern.validate import Validator
>>> config = ConfigObj('test.conf',configspec='default.conf')
>>> validator=Validator()
>>> results = config.validate(validator, copy=True)
>>> results['analysis']['filter']
False

I'll try to understand what the config validator is doing in addition to calling is_string to see why it fails.

[zblz]

OK, found the reason it is failing. The ConfigObj reader parses that string as a list of strings, not a string:

In [9]: config['analysis']['filter']
Out[9]: 
['DATA_QUAL==1&&LAT_CONFIG==1&&ABS(ROCK_ANGLE)<52&&(ANGSEP(RA_src',
 'DEC_src',
 'RA_SCZ',
 'DEC_SCZ)<50)']

[zblz]

The way to avoid the item being read as a list of strings rather than a string is to put quotes (single or double) around the string. Therefore, the config entry:

filter = DATA_QUAL==1&&LAT_CONFIG==1&&ABS(ROCK_ANGLE)<52&&ANGSEP(RA_src,DEC_src,RA_SCZ,DEC_SCZ)<50)

should be changed to:

filter = "DATA_QUAL==1&&LAT_CONFIG==1&&ABS(ROCK_ANGLE)<52&&ANGSEP(RA_src,DEC_src,RA_SCZ,DEC_SCZ)<50)"

@pmunar -- can you confirm that this works?

[pmunar]

Yes, it works!

Thanks @zblz !