UlrikeS91
commented
4 years ago I'm not sure if I agree.
confidence is used in anatomicalEntityRelation, while criteriaQualityType is used in annotation. In annotation, this should describe the quality of the criteria used to produce the annotation. In anatomicalEntityRelation, we were aiming for a measure of how confident we are that the relation between entity-1 and entity-2 is correct.
A binary measurement might be too simplified for that purpose? Also, processive could be tricky in this context (with your definition), too. "You can redo the process that let me to my result" may not always be that easy, but I could still be very confident that entity-1 is related in the way I record to entity-2.
For example, I could get an image of a portion of a brain section with a visible electrode tract located in CA1 of the hippocampus. I can't actually anchor it in QuickNII, because it's only one section and too few additional landmarks to do this properly. But I can clearly see that the electrode is in CA1 because CA1 is a very good landmark on it's own. But this is only easy to spot if you know what to look for. Theoretically, other people could redo this process (even though it would involve studying neuroanatomy extensively), so then it would be processive? I could also have the same example but when the section was mounted there was an air bubble trapped right on the tip of the visible elctrode tract making this portion of the image a bit blurry. Still can't anchor it, still can clearly tell it's hippocampus (because not all of it is blurry), and I'm still pretty sure it's in CA1 but it's just not as obvious as before. There might even be a chance that someone else would say that this conclusion might not be correct. Then this process could potentially not be redone, therefore be tagged with asserted. So, then I either have to live with this conclusion or decrease the granularity and go for the next higher level ("Hippocampus"), which anybody should and would agree. Only then it could be tagged processive.
Do we rather want a simple system or do we want to be able to capture some of these things? I know we said that we don't do provenance tracking with the SANDS schemas, but then we cannot have confidence at all. There is no point in relating entity-1 to entity-2 if I can't add processive. Asserted as in "I say so", has no meaning. Then, I can add an anatomicalEntityRelation to "Hippocampus" relating it to "Cerebellum", and add asserted, which technically wouldn't be wrong. It's only me, who saids so.
I do agree that we could merge them, though. I just disagree to stick to a binary system. Worst is, that I do like asserted as an option, because I have a few example where this would be right on point. Generally, these terms are also more defined. "sure", "very sure" and such are very generic and open.
Maybe we could come up with some additional terms to describe the confidence (that don't clash too much with the purpose of the criteriaQualityType for annotation)? Maybe such a list already exists?
lzehl
When I went over what instances we would actually register for confidence I realized that in principle it's exactly what we describe in with the citeriaQualityType instances:
I'm really not sure if I want to introduce other terms for confidence levels like "sure", "very sure", "maybe" etc. I like the simplicity but explanatory power of processive and asserted.
@UlrikeS91 & @tgbugs Could we just unit those into one property called "criteriaConfidence" or "confidenceOfCriteria" or something like this?