How to track annotation confidence?

[hohonuuli]

Provide confidence levels for annotations before and after data are uploaded to FathomNet Taxonomist should be able to assign a confidence level Person requesting change should provide explanation for suggestion Comment stream for each annotation

track confidence if user has an actual sample of the animal in the photograph.

[claudenozeres]

@hohonuuli, are you looking for examples of how this is done (e.g., on projects in BIIGLE), proposing best practices, or noting is an issue in FathomNet to resolve? Thanks! Claude

[hohonuuli]

@claudenozeres We're open to guidance on this one. This issue was created from notes from user feedback surveys.

[claudenozeres]

Thanks @hohonuuli . I mentioned that because a colleague (Barbara Neves) working with underwater video imagery did a recent ICES presentation on that topic. Will ask to share. Quality control is something we wonder about--going beyond annotations with names.

[hohonuuli]

@claudenozeres Yes, please share the presentation!

[hohonuuli]

We track changes and allow comments now. Is this sufficient?

[hohonuuli]

Do we want to capture the "confidence" from ML models?

[claudenozeres]

Is a step, but I would not say it is adequate. If not clear/obvious, I could expound on this.

[hohonuuli]

@claudenozeres Just a note that we're currently prioritizing based on the amount of engineering resources we have.