Annotation training

[mzur]

It was suggested in the MIW 2017 that there should be a way to train researchers how to produce reliable and accurate annotations:

The attendees agreed that consistent annotation systems are necessary for any type of annotation. [...] Attendees suggested that open access tutorials and training sets could be helpful in improving taxonomic understanding; it was suggested that it would be valuable to have a common platform for sharing and distributing this information (e.g. a webpage providing links to different sources).

This could be the basis for a new BIIGLE module which takes a user defined rule set for annotations ("annotation scheme") and walks users through a tutorial-like training so they can familiarize themselves with the annotation tool and with the types of annotations that are expected of them.

Ideas:

• The annotation schemes should be independent from BIIGLE projects, similar to label trees. They can be attached to a project if annotations on the project should be created according to the annotation scheme.
• Each annotation scheme has a tutorial that trains users how to annotate according to the scheme.
• Users could get achievements/badges if they completed a tutorial for an annotation scheme, encouraging them to complete the tutorials to raise their annotation "proficiency" as seen by other users.
• A tutorial may include tasks like:
  • Finding all objects that should be found (with point annotations)
  • Using the right annotation shape for the right object
  • More??

[mzur]

A basic version can look like this:

An "annotation strategy" (?) can be created for each project. The specification contains a few selected images that are already annotated. These images serve as an example on how to properly create annotations. In addition to that the users can upload a PDF where they can describe their annotation strategy in detail. We could also limit this to the PDF to keep it simple. The users can include screenshots of properly annotated images in the PDF, too, after all. A button to download the annotation strategy will be displayed in the project overview page. The annotation strategy thing will be continued in BiodataMiningGroup/biigle-projects#19.

This issue will continue the discussion on a more generic "tutorial" for how to use Biigle for annotations.

[mzur]

Georges also suggested something like this. His version uses tooltips to guide the user step by step through the annotation tool.

[mzur]

This paper describes how a global standardised marine taxon reference can be developed. We should take this into account when we develop the annotation strategy feature. They aim to provide an online database as reference but it does not exist yet. Maybe BIIGLE can interface with this database when it is created.