FAST-HEP / fast-plotter

Manipulate binned pandas dataframes into plots
https://fast-hep.web.cern.ch
3 stars 8 forks source link

Apply if #71

Closed DBAnthony closed 2 years ago

DBAnthony commented 3 years ago

Add 'apply if' function to postproc stages. Example usage:

add_photon_theory_syst:
  apply_if: "'weight_nominal:sumw' in df.columns"
  evals:
    weight_photon_znn_0b_Down:sumw: '`weight_nominal:sumw` * (0.95 * ((category==''VH_2j0b'') & (process == ''PhotonEwk'')) + ((category!=''VH_2j0b'') | (process != ''PhotonEwk'')))'
DBAnthony commented 3 years ago

Thanks for this David. I think this might be overkill for the original motivation and might make our lives harder for maintability:

  • Using pure python's eval in this way adds a security headache
  • We really break the declarative aspect of this approach
  • We increase the coupling between the main loop and the content of each stage.
  • We need to reserve an extra keyword for every stage ("apply_if") If we had many use cases where this change was needed then we could mitigate the above to some degree, but for the issue that prompted this, I think we can approach things differently by adding a specific argument to that stage to check for the necessary columns.

Yeah that's fair enough. My thinking behind adding it more generally is because there's quite a lot of functions similar to AssignCol where one might expect similar functionality, so thought it made more sense to allow it to be used in any stage rather than for a particular one - as well as it future-proofing any similar situations where for a given stage you want to only apply it in certain cases.

Not sure if you saw what I shared in the mattermost, but I also added an example usage in the original comment here. The way I had it in the config was:

add_photon_theory_syst:
  apply_if: "'weight_nominal:sumw' in df.columns"
  evals:
    weight_photon_znn_0b_Down:sumw: '`weight_nominal:sumw` * (0.95 * ((category==''VH_2j0b'') & (process == ''PhotonEwk'')) + ((category!=''VH_2j0b'') | (process != ''PhotonEwk'')))'
    ...