jhansche
commented
1 year ago Proposal would be an extra argument when strategy=best_guess, alongside confidence_threshold, something like anomaly correction options defining:
max_confidence: if the unrecognized confidence exceeds this max, then assume it is NOT an anomaly. From above scenario, the anomaly had confidence=17, which is unexpectedly high.recognized_threshold: Require at least this percentage of sighting matches to be a recognized species. From above scenario, 8 tokens were recognized successfully and 1 was an anomaly. But there could be other scenarios, like 1 recognized and 1 unrecognized, or even 3 separate sightings having 1 recognized, and 2 separate unrecognized anomalies.sighting_interval- max time interval for all media within a postcard. From above scenario, all media had identical timestamps, which could be misleading. It is possible to have 2 separate birds visiting back to back, where the best guess might actually be correct, and treating it as an anomaly might be wrong.
There's no sure-fire way to address this, and the ideal fix would be for BB to take these datapoints into consideration in order to make these contextual decisions more intelligently. I can account for the one use case I've encountered, but cannot account for everything.
Importantly, this library has no intention of introducing its own AI/ML identification, so proceeding with this enhancement would be best-effort only. And the best solution to this might actually be to use a separate ML classifier service (like Cornell's Merlin Bird ID).
We know that BB only considers each individual photo, by itself, when determining likely species. This opens the possibility for some really wacky decisions. For example:
The bird butt was obviously the same Carolina Wren as the other 8, and the Fish Crow identification is obviously wrong.
Looking at the postcard data (internal notes at
issue-40.json), I think we can work towards being smarter about the one anomalous sighting by checking the media timestamps. If all the media are very close together, yet the media seem to be split into separate sighting instances, one of which has a low-confidence suggestion, while most of the other media have a high-confidence recognized species, we can reasonably infer that the low-confidence one is an anomaly, and assign the other sighting's species to the anomaly.We could also work this behavior into the collect_postcard strategy configs.