IIIF Presentation 2 and annotations (in short)

A manifest does not directly contain annotations
Annotations are externalized in annotation lists at different endpoints
A manifest can contain multiple annotation lists by canvas. In our case:
- One list with a describing motivation (transcription segments)
- One list with a commenting motivation (free annotations)
These two different motivations are grouped through the entire manifest into two different annotation layers
Annotations can be portrayed by shapes using oa:SvgSelector and oa:FragmentSelector (xywh=...) and embed textual content

The existing

The Application

The Adele Application is written in VueJS 2.6 (see github.com/chartes/adele-vue/)
Mirador 3 and the annotation plugin is written in ReactJS.

The application make a use of the IIIF viewer in two different context:

1. The read only mode as seen by visitors and users without right access to a document. In this mode users can see the annotations showing up when they hover the bound shapes with their mouse cursor. The annotation texts are displayed over the image.
1. The edition mode add the ability to create,read,update and delete annotations from the Adele database. Thus they are said to be dynamic annotations.

The API

The work in progress Adele API is written in Python 3 with Flask & SqlAlchemy (sqlite) and serves the manifests fed to Mirador (see github.com/chartes/adele-app/). The annotations are stored in our relational db in two tables:

image_zone: a reference to a particular canvas, coords and an optionnal note associated to the annotation (used for the commenting motivation).
alignment_image: a reference to a image_zone row and two text pointers (start:end) delimiting a segment from the actual transcription. The transcriptions (rich text formatted as xml fragments ) are stored in the transcription table.

The API is accessible through REST endpoints and serve manifests with its IIIF Presentation 2 annotations. Some manifests are local but some could also be served from Gallica.

Fetching a manifest

curl http://localhost:5000/api/1.0/iiif/74/manifest

{
  "@context": "http://iiif.io/api/presentation/2/context.json", 
  "@id": "http://localhost:5000/api/1.0/iiif/74/manifest", 
  "@type": "sc:Manifest", 
  "attribution": "École nationale des chartes", 
  "description": "Saint Jérôme, Chronicon", 
  "label": "Dossier documentaire 74", 
  "license": "http://rightsstatements.org/vocab/NoC-NC/1.0/", 
  "logo": "http://193.48.42.68/loris/logos/logo-enc.png/full/full/0/default.png", 
  "metadata": [
    {
      "label": "Manifest Provider", 
      "value": "École nationale des chartes"
    }
  ], 
  "related": "http://theleme.enc.sorbonne.fr/dossiers/notice74.php", 
  "sequences": [
    {
      "@context": "http://iiif.io/api/presentation/2/context.json", 
      "@id": "https://iiif.chartes.psl.eu/images/adele/dossiers/74/sequence/normal", 
      "@type": "sc:Sequence", 
      "canvases": [
        {
          "@id": "https://iiif.chartes.psl.eu/images/adele/dossiers/74/canvas/f1", 
          "@type": "sc:Canvas", 
          "height": 1184, 
          "images": [
            {
              "@context": "http://iiif.io/api/presentation/2/context.json", 
              "@id": "https://iiif.chartes.psl.eu/images/adele/dossiers/74/annotation/a1", 
              "@type": "oa:Annotation", 
              "motivation": "sc:painting", 
              "on": "https://iiif.chartes.psl.eu/images/adele/dossiers/74/canvas/f1", 
              "resource": {
                "@id": "https://iiif.chartes.psl.eu/images/adele/dossiers/74.jpg/full/full/0/default.jpg", 
                "@type": "dctypes:Image", 
                "format": "image/jpeg", 
                "height": 1184, 
                "service": {
                  "@context": "http://iiif.io/api/image/2/context.json", 
                  "@id": "https://iiif.chartes.psl.eu/images/adele/dossiers/74.jpg", 
                  "profile": "http://iiif.io/api/image/2/level2.json"
                }, 
                "width": 900
              }
            }
          ], 
          "label": "Volume de 178 fol.", 
          "otherContent": [
            {
              "@id": "http://localhost:5000/api/1.0/iiif/74/list/commenting-0", 
              "@type": "sc:AnnotationList", 
              "within": {
                "@id": "http://localhost:5000/api/1.0/iiif/74/layer/commenting", 
                "@type": "sc:Layer", 
                "label": "commenting"
              }
            }, 
            {
              "@id": "http://localhost:5000/api/1.0/iiif/74/list/describing-0", 
              "@type": "sc:AnnotationList", 
              "within": {
                "@id": "http://localhost:5000/api/1.0/iiif/74/layer/describing", 
                "@type": "sc:Layer", 
                "label": "describing"
              }
            }
          ], 
          "width": 900
        }
      ]
    }
  ], 
  "thumbnail": {
    "@id": "https://iiif.chartes.psl.eu/images/adele/dossiers/74.jpg/full/180,/0/default.jpg"
  }
}

Annotation layers urls, annotation lists and annotations url are reachable.

Fetching an annotation layer

curl http://localhost:5000/api/1.0/iiif/38/layer/commenting

{
  "@context": "http://iiif.io/api/presentation/2/context.json", 
  "@id": "http://localhost:5000/api/1.0/iiif/38/layer/commenting", 
  "@type": "sc:Layer", 
  "label": "commenting", 
  "otherContent": [
    "http://localhost:5000/api/1.0/iiif/38/list/commenting-0"
  ]
}

Fetching an annotation list

curl http://localhost:5000/api/1.0/iiif/38/list/commenting-0

{
  "@context": "http://iiif.io/api/presentation/2/context.json", 
  "@id": "http://localhost:5000/api/1.0/iiif/38/list/commenting-0", 
  "@type": "sc:AnnotationList", 
  "metadata": [], 
  "resources": [
    {
      "@type": "oa:Annotation", 
      "motivation": "sc:painting", 
      "on": {
        "@type": "oa:SpecificResource", 
        "full": "https://iiif.chartes.psl.eu/images/adele/dossiers/38/canvas/f1", 
        "selector": {
          "@type": "oa:SvgSelector", 
          "value": "<svg xmlns='http://www.w3.org/2000/svg'><circle cx='37' cy='44' r='30'/></svg>"
        }, 
        "within": {
          "@id": "http://localhost:5000/api/1.0/iiif/38/manifest", 
          "@type": "sc:Manifest"
        }
      }, 
      "resource": {
        "@id": "http://localhost:5000/api/1.0/iiif/38/annotation/24", 
        "@type": "dctypes:Text", 
        "chars": "Ouverture de l'acte : signe hérité des deux points (..) tracé de façon cursive et N capital redoublé"
      }
    }, 
    {
      "@type": "oa:Annotation", 
      "motivation": "sc:painting", 
      "on": {
        "@type": "oa:SpecificResource", 
        "full": "https://iiif.chartes.psl.eu/images/adele/dossiers/38/canvas/f1", 
        "selector": {
          "@type": "oa:SvgSelector", 
          "value": "<svg xmlns='http://www.w3.org/2000/svg'><circle cx='379' cy='61' r='14'/></svg>"
        }, 
        "within": {
          "@id": "http://localhost:5000/api/1.0/iiif/38/manifest", 
          "@type": "sc:Manifest"
        }
      }, 
      "resource": {
        "@id": "http://localhost:5000/api/1.0/iiif/38/annotation/25", 
        "@type": "dctypes:Text", 
        "chars": "Capitale à redoublement, fréquente aussi dans les écritures de la chancellerie royale au même moment. Noter que cet ornement distingue surtout des noms propres et des noms de fonction, comme certains chiffres"
      }
    }, 
    {
      "@type": "oa:Annotation", 
      "motivation": "sc:painting", 
      "on": {
        "@type": "oa:SpecificResource", 
        "full": "https://iiif.chartes.psl.eu/images/adele/dossiers/38/canvas/f1", 
        "selector": {
          "@type": "oa:FragmentSelector", 
          "value": "xywh=586,50,306,52"
        }, 
        "within": {
          "@id": "http://localhost:5000/api/1.0/iiif/38/manifest", 
          "@type": "sc:Manifest"
        }
      }, 
      "resource": {
        "@id": "http://localhost:5000/api/1.0/iiif/38/annotation/26", 
        "@type": "dctypes:Text", 
        "chars": "Deux signes abréviatifs différents (l'usage est confirmé dans le reste du texte) après la lettre \"r\" ; le premier est en fait spécifique à l'abréviation de \"-orum\" (voir aussi \"armorum\" 1.3). Pour le reste, l'usage des abréviations est relativement discret"
      }
    }, 
    {
      "@type": "oa:Annotation", 
      "motivation": "sc:painting", 
      "on": {
        "@type": "oa:SpecificResource", 
        "full": "https://iiif.chartes.psl.eu/images/adele/dossiers/38/canvas/f1", 
        "selector": {
          "@type": "oa:SvgSelector", 
          "value": "<svg xmlns='http://www.w3.org/2000/svg'><circle cx='138' cy='76' r='11'/></svg>"
        }, 
        "within": {
          "@id": "http://localhost:5000/api/1.0/iiif/38/manifest", 
          "@type": "sc:Manifest"
        }
      }, 
      "resource": {
        "@id": "http://localhost:5000/api/1.0/iiif/38/annotation/27", 
        "@type": "dctypes:Text", 
        "chars": "Signe de ponctuation mineur (trait oblique court et fin), qui, entre autres, sert à ouvrir des incises ou à isoler, comme ici, les chiffres des milliers et des centaines, plus bas les livres et les sous"
      }
    }, 
    {
      "@type": "oa:Annotation", 
      "motivation": "sc:painting", 
      "on": {
        "@type": "oa:SpecificResource", 
        "full": "https://iiif.chartes.psl.eu/images/adele/dossiers/38/canvas/f1", 
        "selector": {
          "@type": "oa:SvgSelector", 
          "value": "<svg xmlns='http://www.w3.org/2000/svg'><path xmlns='http://www.w3.org/2000/svg' d='M 582 147 L 580 167 L 582 215 L 598 227 L 609 214 L 610 174 L 609 149 L 595 134 z'/></svg>"
        }, 
        "within": {
          "@id": "http://localhost:5000/api/1.0/iiif/38/manifest", 
          "@type": "sc:Manifest"
        }
      }, 
      "resource": {
        "@id": "http://localhost:5000/api/1.0/iiif/38/annotation/28", 
        "@type": "dctypes:Text", 
        "chars": "Justification droite parfois incertaine : mais noter que, comme dans l'acte royal, le scripteur ne coupe aucun mot en fin de ligne"
      }
    }, 
    {
      "@type": "oa:Annotation", 
      "motivation": "sc:painting", 
      "on": {
        "@type": "oa:SpecificResource", 
        "full": "https://iiif.chartes.psl.eu/images/adele/dossiers/38/canvas/f1", 
        "selector": {
          "@type": "oa:SvgSelector", 
          "value": "<svg xmlns='http://www.w3.org/2000/svg'><path xmlns='http://www.w3.org/2000/svg' d='M 65 313 L 80 304 L 148 299 L 206 300 L 250 310 L 246 319 L 198 327 L 90 325 z'/></svg>"
        }, 
        "within": {
          "@id": "http://localhost:5000/api/1.0/iiif/38/manifest", 
          "@type": "sc:Manifest"
        }
      }, 
      "resource": {
        "@id": "http://localhost:5000/api/1.0/iiif/38/annotation/29", 
        "@type": "dctypes:Text", 
        "chars": "Clôture du texte par un signe et trois longs traits, qui n'occupent pas toute la ligne finale"
      }
    }, 
    {
      "@type": "oa:Annotation", 
      "motivation": "sc:painting", 
      "on": {
        "@type": "oa:SpecificResource", 
        "full": "https://iiif.chartes.psl.eu/images/adele/dossiers/38/canvas/f1", 
        "selector": {
          "@type": "oa:SvgSelector", 
          "value": "<svg xmlns='http://www.w3.org/2000/svg'><path xmlns='http://www.w3.org/2000/svg' d='M 239 384 L 276 385 L 290 357 L 280 333 L 256 324 L 230 335 L 223 363 z'/></svg>"
        }, 
        "within": {
          "@id": "http://localhost:5000/api/1.0/iiif/38/manifest", 
          "@type": "sc:Manifest"
        }
      }, 
      "resource": {
        "@id": "http://localhost:5000/api/1.0/iiif/38/annotation/30", 
        "@type": "dctypes:Text", 
        "chars": "Simple queue de parchemin, sceau disparu"
      }
    }
  ]
}

Fetching an annotation

curl http://localhost:5000/api/1.0/iiif/38/annotation/25

{
  "@type": "oa:Annotation", 
  "motivation": "sc:painting", 
  "on": {
    "@type": "oa:SpecificResource", 
    "full": "https://iiif.chartes.psl.eu/images/adele/dossiers/38/canvas/f1", 
    "selector": {
      "@type": "oa:SvgSelector", 
      "value": "<svg xmlns='http://www.w3.org/2000/svg'><circle cx='379' cy='61' r='14'/></svg>"
    }, 
    "within": {
      "@id": "http://localhost:5000/api/1.0/iiif/38/manifest", 
      "@type": "sc:Manifest"
    }
  }, 
  "resource": {
    "@id": "http://localhost:5000/api/1.0/iiif/38/annotation/25", 
    "@type": "dctypes:Text", 
    "chars": "Capitale à redoublement, fréquente aussi dans les écritures de la chancellerie royale au même moment. Noter que cet ornement distingue surtout des noms propres et des noms de fonction, comme certains chiffres"
  }
}

The use of Mirador 3

The mirador-annotation plugin

The mirador-annotation plugin (Mirador 3) still lacks, sometime by design, many important features. Some of them are already described and commented in GitHub issues.

Store adapters

The plugin uses a store adapter to make CRUD operations on the annotations. A few of them are already implemented on GitHub (see https://github.com/ProjectMirador/mirador-annotations/blob/master/src/SimpleAnnotationServerV2Adapter.js)

The adapter has public methods for CRUD operations that should communicate with the Adele API.

NB The plugin uses the IIIF Presentation 3 vocabulary and features (like Annotation Page), hence the (existing) code to convert IIIF content to and from v2.

The needs

Wanted features (by order of priority)

1. Loading of existing annotations

Any textual annotation within a manifest should be loaded inside the IIIF Viewer. Manifests are fetched from the API and can indifferently include dynamic annotations and hardcoded ones.

2. Annotations display

Image from E. Cancel and Save buttons in https://github.com/ProjectMirador/mirador-annotations/issues/4

In Mirador 3 the textual annotations are presented as a list in a sidebar.

When we select an annotation its shape is then highlighted.
If we select a shape on the canvas, an item from the list is automatically selected.
The motivation can be filtered using a 'Select ...' dropdown in the topbar. By default all motivations should be displayed (no filtering).

We would like to keep these behaviors and add a new one: the textual content of an annotation should be displayed while hovering over its shape. It should be readable in any circumstances (hence a clever choice of text styling).

3. CRUD of dynamic annotations (with free shapes and different motivations)

3.1 Creation

3.2 Read

See above for curl examples

3.3 Update

3.4 Deletion

4. Support of motivations

During creation, a select box should allow the selection of either the 'describing' or 'commenting' motivation. This is crucial since the textual content will come from different sources in both cases.

See A. Motivation Menu in https://github.com/ProjectMirador/mirador-annotations/issues/4

4.1 Describing

The describing motivation binds a segment of the transcription to the annotation zone. The transcription content displayed will be fetched from the API.

The segmentation must be done within a readonly textarea component so the only modification available is the segmentation tool (only one tool : 'select a segment for this annotation').

The segment should be highlighted with a background color in the textarea component as it is in the speechparts editor:

Segments already bound (in other annotations may be highlighted in a dimmer color in order to guide the user where he may want to continue.

Overlapping segments are forbidden: it shouldn't be possible to select a segment overlapping an existing one (the selection should cancel itself).

4.2 Commenting

The commenting motivation is used to add free annotations. A simple textarea should allow the user to enter the textual content.

Image from D: Annotation content textarea in https://github.com/ProjectMirador/mirador-annotations/issues/4

We may not want to allow html content but only raw text.

Advices / Open to suggestions

Communication between Vue and Mirador

Ideally the Vue App should provide pre-authenticated CRUD callbacks with the API url/endpoint already filled in to the mirador-annotation plugin during its creation.

chartes / adele-vue

Mirador #11