Improving the example in the documentation - eventually adding more

[cosimoNigro]

This issue is related to ronswanson's review for the journal of the open source software, specifically to the item:

Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).

I tried to re-run the Intro example myself, but - to be honest - it took me quite some time to understand what the different steps of the tutorial represent. I feel that the explanation can be improved.

Moreover, reading the paper draft, I got the idea that the unique feature provided by this package was that:

one can create a so-called template or table model by evaluating the simulation on a grid of its parameters, and use interpolation on the output which allows for the simulated model to compared with data via the likelihood in a reasonable amount of time.

But in this tutorial (the only one in the documentation) this is not fully shown. We are only shown a simple analytical function that is evaluated on a grid of parameters and whose values vs energy are stored in a database. I understand that the band function is used for simplicity (and that ronswanson might be especially useful for more sophisticated models) but we are not shown how this model database can speed up a statistical analysis leveraging the interpolation of the model outputs that the author discusses. And maybe this is non trivial to show, but the current example does not show the particular feature advertised in the paper.

Maybe it would be good to add a statistical evaluation relying on the models in the database in this tutorial, to show the advantages of using this framework? Or maybe - even better - one could add another tutorial notebook illustrating the speed-up in the statistical calculation with the database models?

Is it possible to create only energy-dependent models - f(E), can other type of models be created? This is not clear.

P.S. I would not use the term "simulation" for a simple analytical function evaluated on a grid of parameters.

[grburgess]

@cosimoNigro I think this could be done.. there are working examples for real simulations (the motivation behind the work) that are unfortunately closed source at the moment as they are from proprietary blazar spectral code that I do not have permission to open-source at the moment.

Would you prefer something like a mock "simulation" code that is interfaced in? Even in the case of the blazer "simulation" code, the actual calls to the evaluation are python hooks... so there is not a giant leap in going from something like Band function or a power law.

[grburgess]

On the issue of showing the template model in a fit, would it just be sufficient to link the associated software documentation showing how these tools can be combined?

[cosimoNigro]

there are working examples for real simulations (the motivation behind the work) that are unfortunately closed source at the moment as they are from proprietary blazar spectral code

I see, then let's say that it does not matter whether the model is physical or analytical.

What I would like to see, though, is a statistical application of the model generated and, if possible, the eventual speed-up introduced by having and the model already evaluated on a grid of data. Can't you apply the band function in the example to some real or mock dataset?

On the issue of showing the template model in a fit, would it just be sufficient to link the associated software documentation showing how these tools can be combined?

To which documentation and tools you are referring to?

[grburgess]

@cosimoNigro I have tried to make the connection between the example and a real simulation clearer. As far as fitting, I could put some mock data, but it would make the repo heavier. I have instead linked to the 3ML docs which already cover both fitting and fitting table models. Would this be ok?

[cosimoNigro]

It is, thank you.