Added validation for the names of label fields in stages output by LLM.
If ground_truth is in the stage string, but it is not in the identified label fields, then find the best matching field to swap in. If we only have one label field, use this. Otherwise, look for the first label field that doesn't have confidence on the first detection on the first sample (this is a heuristic, not perfect, but aggregating over all values for this seems like overkill)
If predictions is in the stage string but it is not in the identified label fields, then find the best matching field to swap in. Do opposite of ground_truth for the confidence - find first label field that has confidence.
Note:
This does not account for differences in handling Detections vs Classification. That is mostly handled downstream, but this will not be perfect. That being said, it does improve performance :)
Added validation for the names of label fields in stages output by LLM.
ground_truthis in the stage string, but it is not in the identified label fields, then find the best matching field to swap in. If we only have one label field, use this. Otherwise, look for the first label field that doesn't have confidence on the first detection on the first sample (this is a heuristic, not perfect, but aggregating over all values for this seems like overkill)predictionsis in the stage string but it is not in the identified label fields, then find the best matching field to swap in. Do opposite ofground_truthfor the confidence - find first label field that has confidence.Note:
DetectionsvsClassification. That is mostly handled downstream, but this will not be perfect. That being said, it does improve performance :)