rickyegeland
commented
2 months ago It sounds to me like the goal is to provide a boolean field that indicates whether the given forecast/observation pair was included in metrics derived downstream, and to have SPHINX metrics and VIVID baseline (default) metrics include exactly the same set of forecasts. Is that correct?
If yes, I recommend naming this boolean attribute something like "included", and not named something specific to VIVID, since it has broader application and meaning.
Regarding the granularity per quantity (all clear, peak intensity, etc.), if within SPHINX the forecast/observation pair can be included in downstream metrics in some quantities but not others, then it must be specified per quantity. Otherwise, I think one field for the dataframe is better, as it is simpler and non-redundant.
Right now vivid does not filter based on the match status of any forecasted quantity. We want vivid to understand that there is filtering based on match status but don't want to have vivid repeat that filtering/logic - since we'd have to keep vivid and sphinx the same on that logic if it ever changed. The best way to move forward here would be to add new fields (like a vivid_display field) to the dataframe telling vivid that it is okay to display that forecast/observation pair. Discussion point: whether we add just one field to the dataframe for this or one field per different match status (all clear, peak intensity (max and onset), start time, end time, probability, threshold crossing, duration, fluence, fluence spectrum,