search

LSSTDESC / rail_base

Base classes for RAIL
MIT License
0 stars 1 forks source link

Evaluator use cases to guide comprehensive flagging system #8

Closed aimalz closed 1 year ago

aimalz commented 1 year ago

We've sort of been using LSSTDESC/rail_base#5 and LSSTDESC/rail_bpz#28 to discuss flags, but I think it's worth making a new issue to collect information on use cases that would inform an actual system for per-galaxy flags within RAIL, i.e. where RAIL provides shortcuts to generate entries to certain flag fields that we know other DESC pipelines want to use, given that the qp.Ensemble.ancil attribute makes it a solved problem infrastructure-wise. For example, based on this LSSTC Slack discussion, we should make sure an estimator's confidence measure (e.g. ODDS) is retained as ancil information in the output PDFs, and there ought to be an evaluator to calculate something like IQR over S/N; to tie it all together for the CL use case, we could provide a flexible flagger stage that applies user-specified cuts on those ancil fields to assign integer flags. tl;dr let's try to gather use cases here to see if we can identify commonalities between what we know how to make and what people need for downstream pipelines that can guide upcoming development within rail.evaluation.

aimalz commented 1 year ago

This isn't really actionable, so we'll continue discussing this informally on Slack.