3D Model based W3C Annotations via annotorious

[DiegoPino]

What?

I like 3D. I used to work with 3D animation in my past live. 3D annotations are a great way of communicating a very complex to describe piece of our currently supported formats. Descriptive metadata in a pure form might fall short talking about a specific point of a Model or in a complex scenario a full scene.

@jbaiter today pointed me to this example (extension via an extra context.jsonld)

EXAMPLE 105: Extension Example 3

{
  "@context": "http://www.w3.org/ns/anno.jsonld",
  "id": "http://example.org/anno91",
  "type": "Annotation",
  "skos:prefLabel": "3d Annotation",
  "body": {
    "type": "TextualBody",
    "value": "I love this part of the model!"
  },
  "target": {
    "source": "http://example.org/models/robot.3d",
    "selector": {
      "@context": "http://example.org/3d/ns/extension.jsonld",
      "type": "ThreeDSelector",
      "x": 1035,
      "y": 245,
      "z": 782,
      "w": 120,
      "h": 180,
      "d": 90
    }
  }
}

From https://www.w3.org/TR/annotation-vocab/ Which solves one of the issues: how to save the annotation itself. This barely differs from our current per ADO approach in 2D

 "ap:annotationCollection": {
        "https:\/\/studio.archipelago.nyc\/cantaloupe\/iiif\/2\/de2%2Fimage-f6268bde41a39874bc69e57ac70d9764-view-f6f34dae-1da3-49bf-9891-bd17c7a3dfbc.jp2\/info.json": [
            {
                "id": "#66a24474-3d61-4cb8-bce0-f220d027d244",
                "body": [
                    {
                        "type": "TextualBody",
                        "value": "Agnes",
                        "created": "2022-03-09T21:32:40.356Z",
                        "creator": {
                            "id": "\/user\/1",
                            "name": "admin"
                        },
                        "purpose": "commenting",
                        "modified": "2022-03-09T21:32:41.076Z"
                    },
                    {
                        "type": "TextualBody",
                        "value": "Handwritten",
                        "created": "2022-03-09T21:32:44.960Z",
                        "creator": {
                            "id": "\/user\/1",
                            "name": "admin"
                        },
                        "purpose": "tagging",
                        "modified": "2022-03-09T21:32:47.104Z"
                    }
                ],
                "type": "Annotation",
                "target": {
                    "source": "https:\/\/studio.archipelago.nyc\/cantaloupe\/iiif\/2\/de2%2Fimage-f6268bde41a39874bc69e57ac70d9764-view-f6f34dae-1da3-49bf-9891-bd17c7a3dfbc.jp2",
                    "selector": {
                        "type": "FragmentSelector",
                        "value": "xywh=pixel:546.5972290039062,1582.4466552734375,394.52777099609375,156.076904296875",
                        "conformsTo": "http:\/\/www.w3.org\/TR\/media-frags\/"
                    }
                },
                "@context": "http:\/\/www.w3.org\/ns\/anno.jsonld"
            },

For the actual implementation I would tap into the Mouse coordinates (2D) that Annotorious provides and use Ray casting (using the camera as source) approach to figure out an intersecting vertex on the model. Then this would require me to draw the same Square (OR EVEN A POLYGON!) probably as a camera facing (using the normals of a plane) that would stick to that vertex during rotations.

Fancier would be to stick (imagine a gift wrapper) the whole surface to the actual surface of the 3D Mesh, but I might have forgotten how to code that!

This could be also a contribution later to a wider IIIF 3D https://t.co/BOW7Yt5klX discussion

[DiegoPino]

Code wise, I have a good starting point here that I could modify for our current JMS Modeler that uses internally ThreeJS

https://github.com/ccnmtl/astro-simulations/pull/1921/files