matteolucchini1 / BHJet

Semi analytical black hole jet model, jetset group, Amsterdam
MIT License
13 stars 7 forks source link

Add sherpa wrapper #52

Open DougBurke opened 2 years ago

DougBurke commented 2 years ago

The primary change is to allow the BHJet model to be used from Sherpa. There's also some support for building an XSPEC interface [*], but I have not got the time/energy to do the useful part of getting the build to work (I don't think it would be hard).

There are a few minor tweaks to the README to make it read better on GitHub when parsed as markdown, and the use of gsl-config rather than hard-coding the values when building the BHJet code.

The default parameter values and ranges are based on bhjet.sl, as pointed out by Sera, but all mistakes are my own.

Is there a Input/ip.dat file that could be included or referred to let users check that they get sensible results.


[*] the Sherpa interface is built on Sherpa's XSPEC interface, so the wrap_xspec.cpp file that I add which should be useful for XSPEC is also useful for Sherpa.

Sherpa installation

This is added to the BHJet/README.mdfile, but installation requires CIAO 4.14, preferably installed with conda, and then you can say

% cd BHJet/sherpa
% pip install .
...
Successfully built bhjet
Installing collected packages: bhjet
Successfully installed bhjet-0.1

Then you can load it into Sherpa and use it as a model with

sherpa In [1]: import bhjet.ui
Adding additive model: bhjet

sherpa In [2]: create_model_component("bhjet", "mdl")
Out[2]: <BHJet model instance 'bhjet.mdl'>

sherpa In [3]: print(mdl)
bhjet.mdl
   Param        Type          Value          Min          Max      Units
   -----        ----          -----          ---          ---      -----
   mdl.mbh      frozen        1e+06            3        3e+10       msun
   mdl.incl     frozen           40            2           80        deg
   mdl.dkpc     frozen         3000            1        1e+06        kpc
   mdl.redshift frozen            0            0            7
   mdl.jetrat   thawed        0.005        1e-07            1      L_edd
   mdl.r_0      thawed           20            2           80        r_g
   mdl.z_diss   thawed         1000           50       500000        r_g
   mdl.z_acc    frozen         1000          300       500000        r_g
   mdl.z_max    frozen        1e+07       100000        1e+09        r_g
   mdl.t_e      thawed          100           10         2000        keV
   mdl.f_nth    frozen          0.1         0.05         0.95
   mdl.f_pl     frozen            0            0           10
   mdl.pspec    thawed            2          1.5            3
   mdl.compsw   frozen            0 -3.40282e+38  3.40282e+38
   mdl.velsw    frozen            1            1           25
   mdl.infosw   frozen            0 -3.40282e+38  3.40282e+38
   mdl.norm     frozen            1            0        1e+24

Implementation note

So wrap_xspec.cpp provides an "XSPEC-compatible" interface using the C (rather than FORTRAN or C++) API - see https://heasarc.gsfc.nasa.gov/xanadu/xspec/manual/XSappendixLocal.html

This wrapper routine, wonderfully named xspec_jetinterp, is meant to be close to the ISIS bhjet_fit routine - i.e. it does both the call to jetmain and then the call to jetinterp, rather than having a Python-level routine analogous to bhfit_jet. This is less open to user tweaks, but easier to handle (and I think required if you wanted to use the model in XSPEC; in Sherpa we could use a similar approach to ISIS).

Unfortunately as I'm not an ISIS user then I appear to have made a mistake somewhere (see next section). In the S-Lang/ISIS call to bhjet_fit(), are the lo and hi arguments always in Angstroms?

Current problems

Importing the Sherpa model "directly" (a "lower-level" interface than users are expected to use) shows that I have not correctly handled the call to both jetmain and then jetinterp, but I haven't got the time to track this down just yet

% python
Python 3.8.12 | packaged by conda-forge | (default, Oct 12 2021, 21:57:06)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from bhjet import BHJet
>>> mdl = BHJet()
>>> for par in mdl.pars:
...     print(f" {par.name:10s} = {par.val}")
...
 mbh        = 1000000.0
 incl       = 40.0
 dkpc       = 3000.0
 redshift   = 0.0
 jetrat     = 0.005
 r_0        = 20.0
 z_diss     = 1000.0
 z_acc      = 1000.0
 z_max      = 10000000.0
 t_e        = 100.0
 f_nth      = 0.1
 f_pl       = 0.0
 pspec      = 2.0
 f_heat     = 1.0
 f_b        = 0.1
 f_sc       = 1.5e-07
 p_beta     = 0.05
 sig_acc    = 0.1
 l_disk     = 0.01
 r_in       = 1.0
 r_out      = 1000.0
 compar1    = 1000.0
 compar2    = 0.0
 compar3    = 3e-10
 compsw     = 0.0
 velsw      = 1.0
 infosw     = 0.0
 norm       = 1.0
>>> import numpy as np
>>> egrid = np.arange(0.1, 0.8, 0.1)
>>> elo = egrid[:-1]
>>> ehi = egrid[1:]
>>> for z in zip(elo, ehi):
...     print(f"  {z[0]:.1f} , {z[1]:.1f}")
...
  0.1 , 0.2
  0.2 , 0.3
  0.3 , 0.4
  0.4 , 0.5
  0.5 , 0.6
  0.6 , 0.7
>>> mdl(elo, ehi)
gsl: interp.c:83: ERROR: x values must be strictly increasing
Default GSL error handler invoked.
Aborted (core dumped)

For the really low-level interface, first get the default parameter values as I'm too-lazy to type them in myself: note that we have 28 parameters because the last-one is the normalization.

>>> from bhjet import BHJet
>>> mdl = BHJet()
>>> pvals = [p.val for p in mdl.pars]
>>> pvals
[1000000.0, 40.0, 3000.0, 0.0, 0.005, 20.0, 1000.0, 1000.0, 10000000.0, 100.0, 0.1, 0.0, 2.0, 1.0, 0.1, 1.5e-07, 0.05, 0.1, 0.01, 1.0, 1000.0, 1000.0, 0.0, 3e-10, 0.0, 1.0, 0.0, 1.0]
>>> len(pvals)
28

Now we can access the "C++ code" (essentially this is equivalent to slirp creating a S-Lang functino that calls the C++ code):

>>> from bhjet._models import xspec_jetinterp

This routtine takes three array/list arguments: the parameters, the low-edge, and the high-edge of the bins (it's done this way so I can just re-use the code we have for interfacing to X-SPEC models, but we could change it if need be):

>>> xspec_jetinterp(pvals, [0.1, 0.2, 0.3, 0.4, 0.5], [0.2, 0.3, 0.4, 0.5, 0.6])
gsl: interp.c:83: ERROR: x values must be strictly increasing
Default GSL error handler invoked.
Aborted (core dumped)

So, am I calling it wrong? The lo/hi edges are assumed to be keV.

DougBurke commented 2 years ago

I've stuck ni sme debugging statements so that the following

import numpy as np
from bhjet import BHJet

egrid = np.arange(0.1, 2.0, 0.1)
elo = egrid[:-1]
ehi = egrid[1:]

mdl = BHJet()
print(mdl)

y = mdl(elo, ehi)

will create

bhjet
   Param        Type          Value          Min          Max      Units
   -----        ----          -----          ---          ---      -----
   bhjet.mbh    frozen        1e+06            3        3e+10       msun
   bhjet.incl   frozen           40            2           80        deg
   bhjet.dkpc   frozen         3000            1        1e+06        kpc
   bhjet.redshift frozen            0            0            7
   bhjet.jetrat thawed        0.005        1e-07            1      L_edd
   bhjet.r_0    thawed           20            2           80        r_g
   bhjet.z_diss thawed         1000           50       500000        r_g
   bhjet.z_acc  frozen         1000          300       500000        r_g
   bhjet.z_max  frozen        1e+07       100000        1e+09        r_g
   bhjet.t_e    thawed          100           10         2000        keV
   bhjet.f_nth  frozen          0.1         0.05         0.95
   bhjet.f_pl   frozen            0            0           10
   bhjet.pspec  thawed            2          1.5            3
   bhjet.f_heat frozen            1            1           50
   bhjet.f_b    frozen          0.1        0.001            1
   bhjet.f_sc   thawed      1.5e-07        1e-09          0.1
   bhjet.p_beta thawed         0.05        0.001            1
   bhjet.sig_acc frozen          0.1         0.01            1
   bhjet.l_disk thawed         0.01       0.0001            1      L_edd
   bhjet.r_in   thawed            1            1          200        r_g
   bhjet.r_out  frozen         1000           10       100000        r_g
   bhjet.compar1 frozen         1000            3        1e+06
   bhjet.compar2 frozen            0            0            1
   bhjet.compar3 frozen        3e-10            0            1
   bhjet.compsw frozen            0 -3.40282e+38  3.40282e+38
   bhjet.velsw  frozen            1            1           25
   bhjet.infosw frozen            0 -3.40282e+38  3.40282e+38
   bhjet.norm   frozen            1            0        1e+24
# DBG: user energy grid ..
  energy[0] = 0.1
  energy[1] = 0.2
  energy[2] = 0.3
  energy[3] = 0.4
  energy[4] = 0.5
 ...
  energy[17] = 1.8
  energy[18] = 1.9
# DBG: egrid for jetmain ..
  egrid[0] = 1e-11
  egrid[1] = 1.1749e-11
  egrid[2] = 1.38038e-11
  egrid[3] = 1.62181e-11
  egrid[4] = 1.90546e-11
 ..
  egrid[295] = 4.46684e+09
  egrid[296] = 5.24807e+09
  egrid[297] = 6.16595e+09
  egrid[298] = 7.24436e+09
gsl: interp.c:83: ERROR: x values must be strictly increasing
Default GSL error handler invoked.
Aborted (core dumped)

which shows the inputs to jetmain(egrid, NBINS, const_cast<double *>(parameter), energ, phot); where it fails with the GSL error. The first range (0.1 to 1.9, which is in keV) are the values of the energ array and the 299 or so bins refer to the egrid input.

matteolucchini1 commented 2 years ago

I'm so sorry for getting to this only now! Unfortunately that particular function goes back to the earliest and most horribly documented days of the code (despite my best efforts...). I had a chat with Mike Nowak about this (since jet_interp is originally his code), I will forward you our email exchange and we can go from there!

DougBurke commented 2 years ago

For reference, an example "session" in Sherpa - using CIAO 4.14 installed with conda and only installing the sherpa-relevant parts

% ciaover
# packages in environment at /home/dburke/anaconda/envs/bhjet:
#
# Name                    Version                   Build  Channel
ciao                      4.14.0           py38h59af02c_0    https://cxc.cfa.harvard.edu/conda/ciao
ciao-contrib              4.14.0                     py_3    https://cxc.cfa.harvard.edu/conda/ciao
sherpa                    4.14.0           py38h3fd9d12_0    https://cxc.cfa.harvard.edu/conda/ciao

System information:
Linux dburke-Precision-5530 5.13.0-23-generic #23-Ubuntu SMP Fri Nov 26 11:41:15 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
% cd BHJet/sherpa
% pip install .
Processing /home/dburke/sherpa/BHJet/BHJet/sherpa
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: bhjet
  Building wheel for bhjet (pyproject.toml) ... done
  Created wheel for bhjet: filename=bhjet-0.1-cp38-cp38-linux_x86_64.whl size=522469 sha256=5d149fbad19587772194a7a15c57c8f4de1ec0abef58a3bbde3917b9e5fba553
  Stored in directory: /tmp/pip-ephem-wheel-cache-z0ssfxek/wheels/24/48/13/afb0e0b6b0c25ab5b591aed8645fd627b67b0d8f5dc7bcbcad
Successfully built bhjet
Installing collected packages: bhjet
Successfully installed bhjet-0.1

we can now create a model instance [I have a custom sherpa prompt]

% sherpa
-----------------------------------------------------
Welcome to Sherpa: CXC's Modeling and Fitting Package
-----------------------------------------------------
Sherpa 4.14.0

Python 3.8.12 | packaged by conda-forge | (default, Oct 12 2021, 21:57:06)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.30.1 -- An enhanced Interactive Python. Type '?' for help.

IPython profile: sherpa
Using matplotlib backend: TkAgg
WARNING: imaging routines will not be available,
failed to import sherpa.image.ds9_backend due to
'RuntimeErr: DS9Win unusable: Could not find ds9 on your PATH'

sherpa-4.14.0> import bhjet.ui
Adding additive model: bhjet

sherpa-4.14.0> create_model_component("bhjet", "mdl")
<BHJet model instance 'bhjet.mdl'>

sherpa-4.14.0> print(mdl)
bhjet.mdl
   Param        Type          Value          Min          Max      Units
   -----        ----          -----          ---          ---      -----
   mdl.mbh      frozen        1e+06            3        3e+10       msun
   mdl.incl     frozen           40            2           80        deg
   mdl.dkpc     frozen         3000            1        1e+06        kpc
   mdl.redshift frozen            0            0            7
   mdl.jetrat   thawed        0.005        1e-07            1      L_edd
   mdl.r_0      thawed           20            2           80        r_g
   mdl.z_diss   thawed         1000           50       500000        r_g
   mdl.z_acc    frozen         1000          300       500000        r_g
   mdl.z_max    frozen        1e+07       100000        1e+09        r_g
   mdl.t_e      thawed          100           10         2000        keV
   mdl.f_nth    frozen          0.1         0.05         0.95
   mdl.f_pl     frozen            0            0           10
   mdl.pspec    thawed            2          1.5            3
   mdl.f_heat   frozen            1            1           50
   mdl.f_b      frozen          0.1        0.001            1
   mdl.f_sc     thawed      1.5e-07        1e-09          0.1
   mdl.p_beta   thawed         0.05        0.001            1
   mdl.sig_acc  frozen          0.1         0.01            1
   mdl.l_disk   thawed         0.01       0.0001            1      L_edd
   mdl.r_in     thawed            1            1          200        r_g
   mdl.r_out    frozen         1000           10       100000        r_g
   mdl.compar1  frozen         1000            3        1e+06
   mdl.compar2  frozen            0            0            1
   mdl.compar3  frozen        3e-10            0            1
   mdl.compsw   frozen            0 -3.40282e+38  3.40282e+38
   mdl.velsw    frozen            1            1           25
   mdl.infosw   frozen            0 -3.40282e+38  3.40282e+38
   mdl.norm     frozen            1            0        1e+24

Let;'s fake a data set:

sherpa-4.14.0> egrid = np.logspace(-3, 2, 50)

sherpa-4.14.0> elo = egrid[:-1]; ehi = egrid[1:]

sherpa-4.14.0> load_arrays(1, elo, ehi, np.zeros_like(elo), Data1DInt)

We can now associate the model

sherpa-4.14.0> set_source(mdl)

NOTE I should have been able to say set_source(bhjet.mdl) but this failed for some reason I need to investigate, but that's all on me.

and the model looks like

sherpa-4.14.0> plot_model(xlog=True, ylog=False)

screenshot

DougBurke commented 2 years ago

Doh, I know why set_source(bhjet.mdl) failed - it's because bhjet is the name of the python module. I need to rename some things...

sherpa-4.14.0> import bhjet.ui
Adding additive model: bhjet

sherpa-4.14.0> [m for m in list_models() if m.find('jet') != -1]
['bhjet', 'xsgrbjet', 'xsjet']

sherpa-4.14.0> type(xsjet)
sherpa.ui.utils.ModelWrapper

sherpa-4.14.0> type(bhjet)
module

that last line should have returned sherpa.ui.utils.ModelWrapper...

DougBurke commented 2 years ago

So, with the following file (renamed hack.cpp), which "just" calls jetmain and jetinterp with what-I-believe-to-be the default values (the parameter values and call to jetinterp is taken from my understanding of bhjet.sl):

hack.cpp.md

If I build this in BHJet/ with

g++ -Wall -fsanitize=address -o hack hack.cpp \
  ../Kariba/BBody.cpp ../Kariba/Bknpower.cpp ../Kariba/Compton.cpp \
  ../Kariba/Cyclosyn.cpp ../Kariba/Kappa.cpp ../Kariba/Mixed.cpp \
  ../Kariba/Particles.cpp ../Kariba/Powerlaw.cpp ../Kariba/Radiation.cpp \
  ../Kariba/ShSDisk.cpp ../Kariba/Thermal.cpp \
  bhjet.cpp utils.cpp jetpars.cpp \
  `gsl-config --cflags` `gsl-config --libs`

then run with

LD_LIBRARY_PATH=${CONDA_PREFIX}/lib ./hack

then the output contains the following, which lists the output of jetinterp:

- called jetinterp
  i= 0 energy_lo=0.1 flux=1.22692
  i= 1 energy_lo=0.2 flux=0.380585
  i= 2 energy_lo=0.3 flux=0.119449
  i= 3 energy_lo=0.4 flux=0.0392025
  i= 4 energy_lo=0.5 flux=0.0111242
  i= 5 energy_lo=0.6 flux=0.00365854
  i= 6 energy_lo=0.7 flux=0.00111635
  i= 7 energy_lo=0.8 flux=0.000373105
  i= 8 energy_lo=0.9 flux=0.00013734
  i= 9 energy_lo=1 flux=3.80373e-05
  i= 10 energy_lo=1.1 flux=1.47477e-05
  i= 11 energy_lo=1.2 flux=2.67524e-06
  i= 12 energy_lo=1.3 flux=1.67615e-06
  i= 13 energy_lo=1.4 flux=6.77059e-07
  i= 14 energy_lo=1.5 flux=3.61782e-07
  i= 15 energy_lo=1.6 flux=1.85877e-07
  i= 16 energy_lo=1.7 flux=nan
  i= 17 energy_lo=1.8 flux=nan
  i= 18 energy_lo=1.9 flux=nan
  i= 19 energy_lo=2 flux=nan
  i= 20 energy_lo=2.1 flux=nan
  i= 21 energy_lo=2.2 flux=nan
  i= 22 energy_lo=2.3 flux=nan
  i= 23 energy_lo=2.4 flux=5.298e-09
  i= 24 energy_lo=2.5 flux=3.99261e-09
  i= 25 energy_lo=2.6 flux=2.68722e-09
...

Here's the full output

hack.out.md

So it's generating NaN's - which suggests I'm doing something wrong calling jetmain/jetinterp

matteolucchini1 commented 2 years ago

This is very very confusing to me still. What happens if you use a wider grid (say, from radio to hard X-rays)? If the issue goes away in that case I would say everything is fine, since the model is not supposed to be used for fitting X-rays only anyway.