IIIF encoding in Humdrum files

[craigsapp]

IIIF image information can be stored in Humdrum digital scores. Here is a demo of how to do it (I have not had time to add to the VHV documentation yet).

(1) Copy the IIIF manifest, such as https://api.digitale-sammlungen.de/iiif/presentation/v2/bsb00075346/manifest from the website with the scanned images, such as this one: https://www.digitale-sammlungen.de/en/view/bsb00075346?page=11

(2) Got to the NIFC annotator website: https://pl-wnifc.github.io/nifc-iiif-annotator

Click on the blue circle with the white plus sign at the top left corner of the page to add a new manifest entry:

Click on "+ ADD RESOURCE" button at the bottom right corner of the page:

And then paste the IIIF link copied previously, which in the example case is:

https://api.digitale-sammlungen.de/iiif/presentation/v2/bsb00075346/manifest

Then click the blue "ADD" button to load the entry onto the website.

Click on entry in the manifest list that you want to extract bounding boxes from, and it will load into the annotator:

Go to the desired page of music from gallery view:

And select the desired page from the list of page thumbnails:

(3) To extract bounding boxes from page, click on the square gray plus button in the Annotation section of the page (which is displayed if you click on the word balloon icon highlighted in blue in the following example:

After clicking on the gray plus button, a control panel should appear on the right side of the image:

Click on the square icon just underneath the the gray cursor button:

Now you can draw boxes in the image, and each box will cause an entry to appear in the text box in the annotation tool:

Then copy the text that is added in the text box on the right, by pressing the blue Copy button:

*xywh:75,57,2846,552
*xywh:104,598,2787,427
*xywh:140,985,2769,445
*xywh:87,1424,2805,404
*xywh:98,1793,2775,499

These are the five bounding boxes for the systems of the example page. As a side-effect of pressing the Copy button, the text will disappear from the text area (by design since it is not possible to edit the text, so it needs to be cleared out for processing the next page).

Take note of the scan number from the bottom of the interface, which is 11 in this case:

And add it as a postfix parameter to the bounding box information:

*xywh-#11:75,57,2846,552
*xywh-#11:104,598,2787,427
*xywh-#11:140,985,2769,445
*xywh-#11:87,1424,2805,404
*xywh-#11:98,1793,2775,499

There is a page label on the right: (0011). This can also be used (as long as it is unique in the list of pages)

*xywh-(0011):75,57,2846,552
*xywh-(0011):104,598,2787,427
*xywh-(0011):140,985,2769,445
*xywh-(0011):87,1424,2805,404
*xywh-(0011):98,1793,2775,499

Then add these interpretations to the score (in the **kern columns for the part at the starting point in the music for each bounding box).

Once the bounding boxes are encoded in the score, you can double click on any note in VHV, and it will open the image for that region of the music in a separate pop-out window:

The image is downloaded at 100% resolution, and you should be able to toggle fitting the image to the window and max pixel view by clicking on the image in the window:

Currently there is not conversion of the IIIF information to MEI, but if you want it, you can create a mock-translation of how the IIIF info should be embedded in the MEI conversion.

[craigsapp]

Here is a full demo of IIIF bounding box embedding in a Humdrum score:

https://verovio.humdrum.org/?file=https%3A%2F%2Fdata.nifc.humdrum.org%2F15xx%3A298.krn

You can click on any note, and the image of the staff on which the note comes from in the source will be displayed.

[craigsapp]

Also, here is the first Github URL for 150 Psalm settings by Mikołaj Gomółka that I used to prototype the IIIF data embedding (this is the same Psalm 1 as your test file, published in the same decade, but in Polish and four parts, and mostly homophonic):

https://github.com/pl-wnifc/humdrum-polish-scores/blob/main/pl-wn/kern/pl-wn--sd-xvi-qu-273--001_gomolka-mikolaj--melodiae-na-psalterz-polski-i-szczesliwy-ktory-nie-byl-miedzy-zlymi-w-radzie.krn

For easier interaction with VHV, here is the link for the first psalm: https://verovio.humdrum.org/?file=https%3A%2F%2Fdata.nifc.humdrum.org%2F15xx%3A206.krn

(No. 2 ends in 207.krn for example).

[craigsapp]

Also I forgot to mention that the manifest needs to be stored in a reference record:

!!!IIIF: https://api.digitale-sammlungen.de/iiif/presentation/v2/bsb00075346/manifest

---

Related to the filled-in notes discussed in issue #1, I need to allow for multiple manifests in the same score. Probably it would be like this:

!!!IIIF-c: https://api.digitale-sammlungen.de/iiif/presentation/v2/bsb00075346/manifest
!!!IIIF-t: https://api.digitale-sammlungen.de/iiif/presentation/v2/bsb00075347/manifest
!!!IIIF-b: https://api.digitale-sammlungen.de/iiif/presentation/v2/bsb00075348/manifest

Where each manifest is given a tag (c, t and b), and then the *xywh interpretations would have a similar tag to indicate which manifest the bounding box info refers to:

*xywh-c-#11:104,598,2787,427
*xywh-t-#11:87,1424,2805,404
*xywh-b-#11:98,1793,2775,499

This could also allow encoding multiple simultaneous bounding boxes, one for each source, but how to easily select the source you want to see when double-clicking on a note in VHV would need more thinking (such as a drop-down box to select a particular source).

[craigsapp]

Also I now remember there is a dedicated repository for the psalm settings:

https://github.com/pl-wnifc/mikolaj-gomolka-psalms

but these are mostly just for testing, with any updates (none that I really know of) done in the repository mentioned above.

[WolfgangDrescher]

This is really nice! Thank you for this hint. A keyboard shortcut would be nice, since it can be a bit fiddly to double click on the notes.

Currently there is not conversion of the IIIF information to MEI, but if you want it, you can create a mock-translation of how the IIIF info should be embedded in the MEI conversion.

That's something for the future for me. First I want to get to know Humdrum better before I venture into MEI.

[craigsapp]

A keyboard shortcut would be nice, since it can be a bit fiddly to double click on the notes.

It would be difficult to have a keyboard shortcut, since each staff (as in your repertory) have a different source image...

The main problem is that half-notes and larger durations have a hollow center that cannot be clicked on. One solution would be to have a transparent box/circle placed underneath the noteheads so that clicking in the center of hollow notes would be able to select them.

A general solution to this problem is solved by Voronoy regions, where a click on the SVG image would select the closest notehead even if not clicking directly on the notehead:

https://github.com/d3/d3-delaunay https://observablehq.com/@mbostock/the-delaunays-dual https://en.wikipedia.org/wiki/Voronoi_diagram

[WolfgangDrescher]

It would be difficult to have a keyboard shortcut, since each staff (as in your repertory) have a different source image...

I meant it you have a note highlighted and then navigate with the arrow keys.

[craigsapp]

I meant it you have a note highlighted and then navigate with the arrow keys.

OK, although at some point you still have to click on a note 😉