lucabaldini / ixpeobssim

Simulation and analysis framework for the Imaging X-ray Polarimetry Explorer
GNU General Public License v3.0
10 stars 13 forks source link

xpbin behaviour #577

Open lucabaldini opened 2 years ago

lucabaldini commented 2 years ago

From the call sequence/doc string of xpbin, i..e

usage: xpbin [-h] --algorithm
{PHA1,PHA1Q,PHA1U,PHA1QN,PHA1UN,CMAP,MDPMAP,MDPMAPCUBE,PCUBE,PMAP,PMAPCUBE,PP,ARMAP,EFLUX,LC,LTCUBE}
[--suffix SUFFIX] [--irfname IRFNAME] [--acceptcorr {True,False}]
[--weights {True,False}] [--weightcol WEIGHTCOL]
[--tbinalg {FILE,LIN,LOG}] [--tmin TMIN] [--tmax TMAX]
[--tbins TBINS] [--tbinfile TBINFILE] [--phasebins PHASEBINS]
[--npix NPIX] [--pixsize PIXSIZE] [--xref XREF] [--yref YREF]
[--proj {AIT,ZEA,ARC,CAR,GLS,MER,NCP,SIN,STG,TAN}] [--emin EMIN]
[--emax EMAX] [--ebinalg {FILE,LIN,LOG,EQP,LIST}] [--ebins EBINS]
[--ebinfile EBINFILE] [--ebinning EBINNING] [--phibins PHIBINS]
[--thetabins THETABINS] [--mc {True,False}]
[--overwrite {True,False}]
filelist [filelist ...]
xpbin: error: the following arguments are required: filelist, --algorithm

one could expect all keywords given to be respected, i.e. if one calls xpbin to make a pulse profile in a given energy range one would naively expect --ebins, --emin etc to be respected, but that’s not actually happening, i.e. xEventBinningBase only does binning but not filtering, so no matter which --emin --emax are supplied the xpbin call with --alghoritm=PP/CMAP/PCUBE etc no energy filtering actually takes place. I list this as a bug because that’s a major issue, we need either to update documentation to explicitly request all required filtering to be done before binning, or modify binning.base code to either do actual filtering or throw exception when incompatible arguments are supplied.

Original url: https://bitbucket.org/ixpesw/ixpeobssim/issues/577 Created on 2022-03-06T10:58:26.398814+00:00

lucabaldini commented 1 year ago

Hi Victor, thanks for reporting this.

This was actually the intended behavior, but I share you concerns about xpbin.py command-line arguments being overly complex and confusing, and I’ll be happy to discuss ways to make things better.

If you invoke the app with the --help command-line switch

[lbaldini@pclbaldini ixpeobssim]$ xpbin.py --help

    Welcome to ixpeobssim 25.6.1 (built on Tue, 01 Mar 2022 10:09:07 +0100).

    Copyright (C) 2015--2022, the ixpeobssim team.

    ixpeobssim comes with ABSOLUTELY NO WARRANTY.
    This is free software, and you are welcome to redistribute it under certain
    conditions. See the LICENSE file for details.

    Visit https://bitbucket.org/ixpesw/ixpeobssim for more information.

usage: xpbin.py [-h] --algorithm
                {PHA1,PHA1Q,PHA1U,PHA1QN,PHA1UN,CMAP,MDPMAP,MDPMAPCUBE,PCUBE,PMAP,PMAPCUBE,PP,ARMAP,EFLUX,LC,LTCUBE}
                [--suffix SUFFIX] [--irfname IRFNAME]
                [--acceptcorr {True,False}] [--weights {True,False}]
                [--weightcol WEIGHTCOL] [--tbinalg {FILE,LIN,LOG}]
                [--tmin TMIN] [--tmax TMAX] [--tbins TBINS]
                [--tbinfile TBINFILE] [--phasebins PHASEBINS] [--npix NPIX]
                [--pixsize PIXSIZE] [--xref XREF] [--yref YREF]
                [--proj {AIT,ZEA,ARC,CAR,GLS,MER,NCP,SIN,STG,TAN}]
                [--emin EMIN] [--emax EMAX]
                [--ebinalg {FILE,LIN,LOG,EQP,LIST}] [--ebins EBINS]
                [--ebinfile EBINFILE] [--ebinning EBINNING]
                [--phibins PHIBINS] [--thetabins THETABINS]
                [--mc {True,False}] [--overwrite {True,False}]
                filelist [filelist ...]

Basic event binning interface.

This application allows to bin event files (i.e., photon lists) in a fairly
generic fashion, creating count spectra, light curves, maps and more advances
data products such as those for storing the Stokes parameters.

Supported binning algorithms
* PHA1: count spectrum
   --mc, weights, weightcol, overwrite, suffix
* PHA1Q: Stokes Q spectrum
   --mc, weights, weightcol, overwrite, suffix
* PHA1U: Stokes U spectrum
   --mc, weights, weightcol, overwrite, suffix
* PHA1QN: normalized Stokes Q spectrum (Q/I)
   --mc, weights, weightcol, overwrite, suffix
* PHA1UN: normalized Stokes U spectrum (U/I)
   --mc, weights, weightcol, overwrite, suffix
* CMAP: count map in sky coordinates
   --mc, npix, pixsize, proj, xref, yref, overwrite, suffix
* MDPMAP: MDP map in sky coordinates
   --mc, acceptcorr, weights, weightcol, emin, emax, npix, pixsize, proj, xref, yref, overwrite, suffix
* MDPMAPCUBE: MDP map cube in sky coordinates and energy layers
   --mc, acceptcorr, weights, weightcol, npix, pixsize, proj, xref, yref, emin, emax, ebins, ebinalg, ebinning, ebinfile, overwrite, suffix
* PCUBE: polarization cube (in energy layers)
   --mc, acceptcorr, weights, weightcol, emin, emax, ebins, ebinalg, ebinning, ebinfile, overwrite, suffix
* PMAP: polarization map in sky coordinates
   --mc, acceptcorr, weights, weightcol, emin, emax, npix, pixsize, proj, xref, yref, overwrite, suffix
* PMAPCUBE: polarization map cube in sky coordinates and energy layers
   --mc, acceptcorr, weights, weightcol, npix, pixsize, proj, xref, yref, emin, emax, ebins, ebinalg, ebinning, ebinfile, overwrite, suffix
* PP: pulse profile
   --phasebins, overwrite, suffix
* ARMAP: rate per unit area map in detector coordinates
   --npix, overwrite, suffix
* EFLUX: energy-flux map in detector coordinates
   --npix, overwrite, suffix
* LC: light curve
   --tmin, tmax, tbins, tbinalg, tbinfile, overwrite, suffix
* LTCUBE: Livetime map cube in sky coordinates and off axis angle bins
   --npix, pixsize, proj, xref, yref, overwrite, suffix

positional arguments:
  filelist              path(s) to the input file(s)

optional arguments:
  -h, --help            show this help message and exit
  --algorithm {PHA1,PHA1Q,PHA1U,PHA1QN,PHA1UN,CMAP,MDPMAP,MDPMAPCUBE,PCUBE,PMAP,PMAPCUBE,PP,ARMAP,EFLUX,LC,LTCUBE}
                        the binning algorithm (default: None)
  --suffix SUFFIX       suffix for the output files (default: None)
  --irfname IRFNAME     name of the response functions to be used (default:
                        None)
  --acceptcorr {True,False}
                        enable/disable the acceptance correction for
                        polarization cubse and maps (default: True)
  --weights {True,False}
                        Use weights for the analysis (default: False)
  --weightcol WEIGHTCOL
                        name of the weight column to be used (default: W_MOM)
  --tbinalg {FILE,LIN,LOG}
                        time binning specification (default: LIN)
  --tmin TMIN           minimum time in s (default: None)
  --tmax TMAX           maximum tims in s (default: None)
  --tbins TBINS         number of bins for LIN/LOG time binning (default: 100)
  --tbinfile TBINFILE   path to the optional time bin definition file
                        (default: None)
  --phasebins PHASEBINS
                        number of bins for phase binning (default: 50)
  --npix NPIX           number of pixels per side in the output image
                        (default: 200)
  --pixsize PIXSIZE     the pixel size of the output image in arcseconds
                        (default: None)
  --xref XREF           the horizontal position of the image center (default:
                        None)
  --yref YREF           the vertical position of the image center (default:
                        None)
  --proj {AIT,ZEA,ARC,CAR,GLS,MER,NCP,SIN,STG,TAN}
                        coordinate projection (default: TAN)
  --emin EMIN           minimum energy in keV (default: 2.0)
  --emax EMAX           maximum energy in keV (default: 8.0)
  --ebinalg {FILE,LIN,LOG,EQP,LIST}
                        energy binning specification (default: LOG)
  --ebins EBINS         number of bins for LIN/LOG energy binning (default: 4)
  --ebinfile EBINFILE   path to the energy bin definition file (default: None)
  --ebinning EBINNING   list containing the bin edges (default: None)
  --phibins PHIBINS     number of bins for LIN/LOG phi binning (default: 75)
  --thetabins THETABINS
                        number of bins for off-axis angle binning (default:
                        17)
  --mc {True,False}     use Monte Carlo information (default: False)
  --overwrite {True,False}
                        overwrite existing output files (default: True)

you will notice that, in the full help, the PP binning algorithm only supports a handful of arguments (those are listed at the top of the help for each algorithms separately).

All the energy-related command-line arguments are applicable to algorithms binning in energy, and are not intended for filtering---I guess what I always had in mind since the beginning was that one would use xpselect for that.

I see at least three things that could be done here:

Of course what you write begs the question of whether we want to support pulse profiles and light curves binned in energy. This might actually be a sensible thing to do, and if you have a use case for that, I’d be happy to implement it. (In which case we might discuss the format.)

Feel free to continue commenting on this thread!

lucabaldini commented 1 year ago

Dear Luca,

Recently, I tried to use LC and CMAP functions in xpbin to produce light curves and count maps in 2— 8 keV. After communications with Riccardo, we found out that the energy cut cannot be done in these two algorithms, although some other algorithms do use the input emax and emin values.

I used the following command:

xpbin --algorithm=LC ${prefix}du1.fits --tbins ${tbins} --overwrite=True —emin=2 —emax=8

There was no error reported and the header history shows that emin and emax values have been applied. It would be great if there is a warning or error about the unsuccessful energy cut.

I was using ixpeobssim 28.0.0. Maybe this problem has been addressed in new version.

Best, Ping

lucabaldini commented 1 year ago

I fully acknowledge that the xpbin interface is complex and can be confusing. If you run xpbin --help you will have a precise list of which parameters are accepted by which algorithm, and you will see that LC and CMAP, indeed, do nothing with emin and emax.

(My reasoning, back at the time, was that one would do xpselect to apply the energy cut first, and then use LC and/or CMAP on the pre-processed file, but I can see why one would expect to be able to do things in one step. I might now even see the use case for a CMAPCUBE algorithm, where one does count maps in energy layers.)

The point about the HISTORY is an interesting one: in my mind that's simply the full command that you typed from the command line, but I can see why one would think that, since emin and emax appear in there, a user could assume that the parameters have been applied.

lucabaldini commented 1 year ago

I would say the most reasonable course of action, here, would be: 1) exit with an error message if the user passes an argument that is not supported 2) discuss if and how we want to expand the current algorithms and or create new ones (e.g. CMAPCUBE)

The first can probably happen fairly quickly, while 2) might take some more time and planning?