icecube / pisa

Monte Carlo-based data analysis
http://icecube.github.io/pisa/
Apache License 2.0
19 stars 47 forks source link

MCEq doesn't have Honda systematics #400

Closed jllanfranchi closed 3 months ago

jllanfranchi commented 6 years ago

See comment from @steven-j-wren here: https://github.com/jllanfranchi/pisa/pull/394#issuecomment-335250536

In summary: "standard honda.py systematics should be included in mceq.py. I think this could be as simple as importing them from honda.py but I'm not sure exactly what set of maps is output by mceq.py"

ShiveshM commented 6 years ago

I had a chat about this with @ts4051 about this at Berlin, and I just want to share my thoughts.

MCEq has much more flexibility for the configurations you can run with and also for the output, e.g. it is capable of doing things like generating different flux predictions for the kaon/pion, conventional/prompt etc. or being able to directly change the primary flux, or the hadronic interaction model. So while a direct port of the Honda systematics implementation would be good, I think that the more contemporary way to implement the systematics here is by using the more advanced MCEq features, although this will take some work....

steven-j-wren commented 6 years ago

I won't deny that MCEq will give you more flexibility with the systematics. However, the standard Honda ones are (to the best of my memory): the index change, the nue/numu ratio, the nu/nubar ratio and the energy scale. Is it correct to say that MCEq will give us systematics which supersede all of these, or simply augment them? Either way I would argue that porting the Honda systematics should be fairly trivial, and then that gives a user the option to just leave them fixed if they have no use for them. Just like in the osc stages we specify all of the oscillation parameters but only actually vary 3 (or even 2) of them.

ShiveshM commented 6 years ago

I agree!, porting the Honda systematics is the way to go, like you said it's easy to do and it gives the user the option. Your question, some of the MCEq systematics are simply augmented like the ratios, index. For the energy scale I think MCEq can supersede the way things are done in Honda, since you are able to do variations on things like the hadronic model.

steven-j-wren commented 6 years ago

Ah I had misunderstood you. I thought you were saying that MCEq won't have need for the simpler systematics.

ts4051 commented 6 years ago

Sorry to be late to the party. For MCEq, we can directly vary the parameters of the cosmic ray flux and hadronic models. My thinking currently is to use these directly as systematics, rather than using derived parameters in the nu flux itself.

In terms of the hadronic uncertainties, it looks like the difference between different hadronic models is small compared to uncertainty in the inputs to these models (e.g. meson-nucleus cross sections at the relevant E_primary/x space). The "blocks" in E_primary/x space (for kaons and pions) from the Barr 2006 paper are implemented in MCEq, and ~4 of them dominate for the O(10-1000) GeV nu flux. So floating just these as systematics (plus some cosmic ray spectrum parameters) is currently what I'm thinking (this is also what Summer has tried in oscFit).

So my plan at least is not to try and use the Honda systematics or the Barr parameterisations from Juan Pablo with MCEq, but instead just use the MCEq inputs directly. I'm not saying that people shouldn't use systematics in any other way, but wanted to share my plan at least!

steven-j-wren commented 6 years ago

I definitely agree that y'all should be moving to more physically motivated systematics and that MCEq is a fantastic way to do that. I'm slightly concerned that it will end up just checking between different discrete models. In my opinion this defeats the point of moving to more physically motivated systematics since then there's no clear reason why Model X is better than Model Y without a proper understanding of the various assumptions and caveats. Though that doesn't sound like what you currently have in mind so perhaps I'm worried for nothing! Haha.

LeanderFischer commented 3 months ago

Honda systematics? Nononono :no_bell: