Collect and curate example analysis use cases with some existing reference implementation

[BenGalewsky]

Assumptions

1. Expressed as user stories
2. Existing reference implementation representative of analysis implementation pre-IRIS-HEP.
3. Reference implementation may be restricted due to policies of the collaborations, or may be modified in some way for practical purposes.
4. User stories should focus on complementary aspects of analysis systems.

Acceptance Criteria

1. User stories should provide some minimal description of the task and refer to reference implementation.

[masonproffitt]

I'm confused a bit by this milestone/issue, since there's this entry that keeps propagating in the AS biweekly meeting notes:

• Collect and curate example analysis use cases with some existing reference implementation (Mason)
  • Several different folks have created individual repos implementing the “8” challenges
  • Mason will create an umbrella github repro.

Should the benchmarks/challenges repo be considered part of this milestone? Or does it belong under something else? (A separate project?)

[cranmer]

Collect and curate example analysis use cases with some reference implementation

User stories:

• Event Processing (left blue box on this page)
• Template Fitting (center box)
• Reinterpretation (right box)
• Analysis evolution cycle: on-boarding / forward evolution / preservation

Benchmark examples (concrete examples with reference implementation)

• Z-peak

  • Reference: Coffea(awkward)+Service X (single-threaded from a single file with uproot: zpeak.ipynb, talk at HOW2019)

  • TSelector (exists and one-to-one with awkward capabilities: ZPeak.h, ZPeak.C)

  • RDataFrame (this one is more basic than the awkward and TSelector based ones, not a one-to-one comparison)

  • ATLAS Run3 EventLoop style (possible)

• KubeCon CMS OpenData Higgs Demo:

  • Reference: Containerized Root analysis shown at KubeCon

  • Awkward (to do)

• A template fit with control regions and systematics (based on a ttbar example)

  • Need to check on the status of reference implementation for ttbar example. Alex did TRex fitter in top context, so should be good

  • Idea that evolved at the meeting: implement a fitting service (eg. send pyhf specification, run it on a GPU or TPU-enabled resource)

• Analysis evolution cycle :

  • Examples in ATLAS using GitLab CI compatible with CERN Analysis Preservation, etc. (multi-b analysis)

[cranmer]

Before marking as done:

• Would like to have more concrete pointers to the reference implementations.

  • Would like to have more concrete pointers to the user stories.

[cranmer]

Links to Analysis use cases:

• [ ] Z-peak

  • @gordonwatts and @masonproffitt @BenGalewsky would like a markdown file describing Z-peak example in as-user-facing repo
  • Kyle adding: https://github.com/iris-hep/as-user-facing/blob/master/z-peak.md

• [ ] CMS-open data (Higgs analysis shown at Kubecon):

  • https://github.com/iris-hep/as-user-facing/blob/master/cms-higgs-open-data.md Need to identify someone to at least partially implement this in Coffea. IRIS-HEP Fellow?

• [ ] Template fit:

  • https://github.com/iris-hep/as-user-facing/blob/master/template-fit.md

• [ ] Reinterpretation:

  • @matthewfeickert could you make a 1-page description of the multi-b recasting example https://github.com/iris-hep/as-user-facing/blob/master/reinterpretation.md

[cranmer]

@gordonwatts @masonproffitt @BenGalewsky waiting on markdown file describing Z-peak in the as-user-facing demo. Similar to the template-fit.md and reinterpretation.md files.

[Edit] I made something for now. Feel free to edit: https://github.com/iris-hep/as-user-facing/blob/master/z-peak.md

[cranmer]

I added one for H->4l https://github.com/iris-hep/as-user-facing/blob/master/cms-higgs-open-data.md

[cranmer]

@jpivarski can you provide a pointer to these things mentioned earlier (I believe from your verbal comment during one of the AS meetings)

Reference: Coffea(awkward)+Service X (single-threaded from a single file with uproot: zpeak.ipynb, talk at HOW2019)

TSelector (exists and one-to-one with awkward capabilities: ZPeak.h, ZPeak.C)

RDataFrame (this one is more basic than the awkward and TSelector based ones, not a one-to-one comparison)

[jpivarski]

Perhaps we have too many messages flying around. I've written notes on this somewhere, but I'm not sure where. Anyway, I found the directory where the code is located, and searching on this URL can find the other things that I've written.

https://github.com/nsmith-/coffea/tree/master/notebooks

(The files are in this directory.)

[cranmer]

ServiceX + TRexFitter extends scope of IRIS-HEP tools for template fit user story https://github.com/kyungeonchoi/ServiceXforTRExFitter