Open luutuntin opened 5 years ago
Hi Alex,
You can use the merging script provided by proycon to merge annotations from different annotators into one file, then make changes on it in FLAT.
We don't really have an interface for adjudication yet in FLAT, this is mostly deferred to application-specific postprocessing scripts, such as the foliaagreement
tool (just a computation without interface though) or through foliamerge
as @OsmanMutlu suggested. Sometimes you can also leverage the power of FLAT to do explicit corrections on annotations, if appropriate for your project.
Thank you, Osman and Maarten. I got the idea now.
I tried foliamerge
(folia-2.1.4 and folia-tools-2.1.6) and received the following error:
File "d:\users\luutuntin\anaconda3\envs\py36\lib\site-packages\foliatools\foliamerge.py", line 41, in mergechildren if isinstance(e, (folia.AbstractTokenAnnotation, folia.AbstractAnnotationLayer)) and parent.id and not hasattr(e, 'merged'): AttributeError: module 'folia.main' has no attribute 'AbstractTokenAnnotation'
I couldn't find foliaagreement
in FoLiA tools, I tried foliaeval
(but I was not sure what is the correct way to provide the value for TYPE argument, e.g. POS or pos-annotation?) and received the following error:
File "d:\users\luutuntin\anaconda3\envs\py36\lib\site-packages\foliatools\foliaeval.py", line 380, in main evaluation = evaluate(docs, Type, foliaset, False, args.corrections, args.confusionmatrix, args.verbose) UnboundLocalError: local variable 'Type' referenced before assignment
Did I ignore certain important details?
I tried foliamerge (folia-2.1.4 and folia-tools-2.1.6) and received the following error.
Hmm, indeed something broke when the new library came out. I just committed a fix and will do a new release.
I couldn't find foliaagreement in FoLiA tools, I tried foliaeval (but I was not sure what is the correct way to provide the value for TYPE argument, e.g. POS or pos-annotation?) and received the following error:
Oops, I indeed meant foliaeval. The type value is equal to the FoLiA XML tag, so in this case pos
.
Hi Maarten,
This relates to #112. In addition to IAA calculation, we need an adjudication interface to compare different annotations and create a gold standard. What are the adjudication practices of FLAT-based annotation projects that you are aware of?
Best, Alex