How to model images and how to integrate images URLs?

[chin-rcip]

For the moment we haven't found any solution to model an pattern that depicts images, and makes a difference between the URL of the image, the digital image, the original physical image if it exists, and the link to the actor.

[Habennin]

I am currently working on the Pharos project, a consortium of photo archives. The key question is modelling images. Thus far, we are pretty satisfied with our modelling. In brief the idea is that you model the basic entity as E36 Visual Item and then from there you indicate its materializations which are either connected through p128 to an E22 or p165 to a D1 digital object. A URI is just an instance of E42 with a type 'URI'.

[stephenhart8]

If I follow what you have done with the Pharos project, we would have the following model:

My questions:

1. What is the difference between the property P128 carries and P65 shows visual item, that seems here more appropriate?
2. The actual URL would be a literal linked to the E41 Appellation with the property P190 has symbolic content?

[Habennin]

1. p65 is just a specialization of p128 to say that what is carried is an image.

2. Yes. there is a dodgy issue here in that a URL and a URI are very close in form so that in fact your URI for your appellation node might be www.something.org/1 and the value put in the RDF literal will also be www.something.org/1. This is redundant, but unavoiable if you want to be able to do things with the appellation node like type it.

[stephenhart8]

To document the source of the image (where the image comes from), could we add dct:source to the D1 Digital Object?

[illip]

I don't see any reason why we couldn't use dct:source except if we decide to use a CRM pattern like P67i -> E73 -> P2 -> E55 (Source).

That said, I would like to keep this issue open to get a better understanding of the issue. Three items in particular:

1. Are we happy with the CRM pattern for the image URL or we would prefer something like schema:contentURL?
2. Do we all agree that our digital images cannot be deleted because they are conceptual objects? I'm still not convince of this semantic choice?
3. Where do we draw the line between one Visual Item and another? Since the Visual Item is the only link between our E24 and D1, it's important to be sure that they will never be divided according to a visual issue like: "Oh I don't think it's the same visual item since I can't see the textures on the digital object."

[stephenhart8]

The pattern I proposed on the 12th of December has been added to the TM 2.0 and will be tested in the following months. It has been decided on the 20th of December to leave this issue open in order to continue the discussion, and see if any other pattern seems better.

[illip]

To help us understanding the E38_Visual_Item, this Linked Art discussion might be useful: https://github.com/linked-art/linked.art/issues/289

[Habennin]

what is the semantic meaning of source? Does it mean someone who owns or provides the digital image?

That said, I would like to keep this issue open to get a better understanding of the issue. Three items in particular:

Are we happy with the CRM pattern for the image URL or we would prefer something like schema:contentURL?

would you put this directly off the image? By creating an instance of D1 Digital Object, you recognize that the digital image has a separate identity, which is actually true.

Do we all agree that our digital images cannot be deleted because they are conceptual objects? I'm still not convince of this semantic choice?

A copy of the digital object could be deleted, but not the pure idea of the digital object until all copies of it are destroyed. Would you track that anyway?

Where do we draw the line between one Visual Item and another? Since the Visual Item is the only link between our E24 and D1, it's important to be sure that they will never be divided according to a visual issue like: "Oh I don't think it's the same visual item since I can't see the textures on the digital object."

If you don't have the means to distinguish them because of lack of documentation, assume they are the same, to make life feasible?

[illip]

Hi @Habennin,

what is the semantic meaning of source? Does it mean someone who owns or provides the digital image?

Good question, I think we would like to capture both information. It will be important to distinguish both in order to manage properly the rights. @stephenhart8 what do you think?

Are we happy with the CRM pattern for the image URL or we would prefer something like schema:contentURL?

would you put this directly off the image? By creating an instance of D1 Digital Object, you recognize that the digital image has a separate identity, which is actually true.

I totally agree that we need a specific node to represent the digital object. :)

Do we all agree that our digital images cannot be deleted because they are conceptual objects? I'm still not convince of this semantic choice?

A copy of the digital object could be deleted, but not the pure idea of the digital object until all copies of it are destroyed. Would you track that anyway?

So in this case, do we need another node to represent a specific digital object? In other words, the URL is a way to identify a location (no worries, I don't want to open a E53_Place discussion here) of a specific digital representation. I know that we can attach several appellations to an entity but in this case, I'm not sure the URL apply to the purely conceptual object, no? I guess I could colorize a specific digital object with its own URL that will perhaps refer to the purely conceptual one and even if I colorized my version, it still incorporates the same visual item. Perhaps, I'm completely misunderstanding here, but for me, it's not clear which instances should be under "D1_Digital_Object".

Where do we draw the line between one Visual Item and another? Since the Visual Item is the only link between our E24 and D1, it's important to be sure that they will never be divided according to a visual issue like: "Oh I don't think it's the same visual item since I can't see the textures on the digital object."

If you don't have the means to distinguish them because of lack of documentation, assume they are the same, to make life feasible?

I still think this is something we should state clearly because if someone thinks that he should create a distinct E36 according to whatever reason we won't be able to connect E24 and D1. So I guess we should explain clearly that when you have a digital version of something physical we should always use the same E36_Visual_Item for both and this is how our mapping process will proceed. That said, I don't think we will have this kind of issue now because the E36 won't appear in our checklist/Reference.

[stephenhart8]

In order to facilitate the discussions on this issue, I have summarized the different questions that needs to be answered:

1. À partir de quand existe-il une distinction entre deux E36 Visual Item? Si un artiste retouche une oeuvre, crée-il un nouveau E36 Visual Item? Si oui, ce nouveau E36 Visual Item contient-il le E36 Visual Item de l'euvre originelle?

   What is the difference between two E36 Visual Item? If an artist transform an already existing image, does he create a new E36 Visual Item? If so, does this new E36 Visual Item contains the E36 Visual Item of the original image?

2. D'une même manière, quelle est la différence entre deux D1 Digital Object? Est-ce que une résolution d'image différente implique la nécessité de créer un nouveau D1 Digital Object?

   Similarily, what is the difference between two D1 Digital Object? Does a different image resolution implies a new D1 Digital Object?

3. Est-ce que P138 represents entre le E36 Visual Item et le E39 Actor est suffisante? Ou faut-il avoir plus de précision dans la description de l'image?

   Is the property P138 represents linking the E36 Visual Item to the E39 Actor is enough for describing actors, or should there be a greater precisious in the description of images?

4. Devrait-on utiliser une autre ontologie pour documenter l'URL d'une image, comme schema:contentURL?

   How to model the URL of a digital image? Should we use a simple non-CIDOC ontology, like schema:contentURL?

5. Si l'on utilise l'ontologie CIDOC CRM, comment représenter les URL des images numériques? Faut-il utiliser la classe E41 Appellation (car la classe E45 Address a été dépréciée) ou E42 Identifier?

   If we decide to use CIDOC CRM, how to model URLs? Should we model them as E41 Appellation (as E45 Address has been deprecated) or as an E42 Identifier?

6. Selon CIDOC CRM, l'URL de l'image devrait elle être le contenu de la classe E41/E42 ou être l'URI de cette classe?

   Following CIDOC CRM, where should the URL of the image be? Should it be the content of the E41/E42 Class or the URI of that class?

7. Est-ce que l'image numérisé doit être considérée comme un objet physique ou plutôt un objet conceptuel? En effet, la classe D1 Digital Object est rdfs:subClassOf de E28 Conceptual Object. Cela signifie que conceptuellement, l'image numérique ne peut pas être détruit. Est-ce problématique?

   Should the digital object be considered a physical object or a conceptual one? The class D1 Digital Object is rdfs:subClassOf of E28 Conceptual Object. This means that a digital object cannot be destroyed. Could this be problematic?

8. Dans le modèle cible 2.0, il n'y a pas de sources associées à l'image. Quelle type de sources devrait-on considérer incorporer? Faut-il mentionner l'acteur qui a fournie l'image, ou l'acteur qui détient les droits? Les deux sont-ils nécessaires? Il me semble que le fournisseur de l'image est déjà documenté dans le Named Graph

   In the Target Model 2.0, there is no source associated with the image. What type of source do we want to have: the actor who provided the image or the actor who holds the rights of the image? It seems that the provider is already mentionned in the Named Graph.

9. Comment documenter la source d'une image? Est-il possible de recourir à l'ontologie DublinCore, notamment la propriété dct:source? Ou faut-il rester au sein de l'ontologie CIDOC CRM?

   How to document the source of an image? Should we use the ontology CublinCore, with the property dct:source? Or should we stay within CIDOC CRM?

[stephenhart8]

From the discussions of the Semantic Committee on the 4th of February 2021, it has been decided that:

• The distinction between different instances of E36_Visual_Item is made by domain experts. The model must allow the creation of links between instances of E36_Visual_Item and this should be added to the Target Model. One option is to link the Visual Item that is incorporating another Visual Item with the property P165_incorporates, as follows:
• The description of Actors depicted in E36_Visual_Item is insufficient in the Target Model 2.0. The property P138_represents does not allow to document how the actor is represented. It is, therefore, necessary to change the pattern. Few options have been proposed: Using the P138.1_mode_of_representation to document how the actor is represented, or by documenting the event the E39_Visual_Item depicts. In order to choose the appropriate pattern, it is first necessary to list the different fields used to document the mode of representation of an actor in an image. This will be done for the next Semantic Committee meeting on the 4th of March.

[marielmat]

As requested on February meeting, here are some examples of items that represent people and the description of these items.

The information about the "subject" can be found in several fields of our database. For the purpose of this specific issue, I chose examples where the subject represented is described under "Title" and / or "Description".

"Description" correspond to Info-Muse definition. In our database, we "share" this field with "Scope and content" (from the RDDA) for the archives description. The quality of the descriptions is uneven.

2006-981, 2008-1716, 1993.15019, PH2018-198

[stephenhart8]

Thank you @marielmat for the uses cases

From what I have collected, the information pertaining to the descriptions of actors within images is as follows:

2006-981

Portrait en buste représentant Nicolas Vincent de face vêtu du costume traditionnel des Hurons-Wendats orné de médailles, d'épaulettes ainsi que de brassards.

2008-1716

Peinture polychrome à l'huile sur toile rectangulaire. Portrait en pied de soeur Marie-Bertille de l'Eucharistie en costume de religieuse regardant de face et tenant un crucifix dans ses mains. Paysage à l'arrière-plan et bouquet de lys à l'avant plan. Cadre de bois peint avec inscriptions et armoiries.

1993.15019

Tirage 3/4.

PH2018-198

Photographie de groupe d'élèves et de prêtres prise à l'extérieur lors de la fête de la Saint-Louis. Monseigneur Paul-Eugène Roy est assis au centre de la première rangée.

Those descriptions are full text, and does not describe more semantically the representation of actors. But as highlited, some common descriptors can be gathered and a few fields could be created:

• The "shot" of the representation (term coming from filmmaking). It is to say if the actor represented is tatally visible ("portrait en pied"), or partly ("en bust", etc.)
• The orientation of the actor. It is to say if an actor is viewed in front ("de face", on the side ("3/4"), etc.
• The localisation of the actor within a Visual Item. This can be distinguished between the localisation in regard of the visual item (on the left of the image, etc.) or in regard of other elements within the visual item (on the left of person X, etc.)
• The interaction with other entities (costume weared, object held, etc.). The description of this interaction will vary depending on whether the other entity is also documented in a museum. This is the case if an object depicted is also in the museum collection. If it is the case, the description of the interaction can be made in the pattern of the object depiction.

There is two ways to of describing images, both that can be used at the same time:

• With a text field, as it is done at the Musée de la Civilisation
• With some vocabulary terms in a structured maner

The text field can be done with the use of an E33_Linguistic_Object associated to the E36_Visual_Item.

The more structured description of the representation of an actor through vocabulary terms is more difficult. One way is to use the property P138.1_mode_of_representation, linking the class PC138_Represents with an E55_Type. The scope note of P138.1_mode_of_representation is ambiguous:

“P138.1 mode of representation allows the nature of the representation to be refined” p. 123

It is unclear if this property can be used for the description of actors within visual items, or if it should be used the medium (for example that an actor is represented painted, or photographed, etc.).

My proposition for the description of actors in visual items is as follows:

Questions

• Can the property P138.1_mode_of_representation be used to describe how an actor is represented or should we use another property?
• A property P67.1_has_type, linking the property P67_refers_to to an E55_Type, super-property of P138_represents could be used, but the Property Class RDF does not indicate that PC138_represents is a sub-class of PC67_refers_to. This P67.1_has_type as a scope more general than P138.1
• How to describe vague concepts as localisation within images with vocabulary terms?
• In my proposition, there is no way to link an object to an actor within a visual item. Therefore, if two actors wearing two different costumes, it would not be possible to know who is wearing which. Is there a way to remedy this, or should we rely. on the textual description?
• Are there other projects who thought about this issue? I've checked in the Linked.art. and SARI models, as well as the other extensions of CIDOC CRM, and I saw no better way of describing visual items. Instead of re-inventing the wheel, the best would be to rely on other projects

[Christian-McCord]

Les descripteurs utilisés par le Musée McCord afin de décrire une personne dans une œuvre ont été développés à une époque où les bases de données n’intégraient pas d’images. Le musée adhère au RCIP en 1988 et entreprend le catalogage de ses collections avec le logiciel Advanced Revelation, en version DOS. Les champs Title (différents types), Subject et Description qui servent à identifier et à décrire les personnes compensent l’absence d’une image en étant le plus descriptif possible. Nous utilisons encore ce modèle aujourd’hui avec The Museum System malgré la possibilité de lier une image à l’enregistrement. D’autres champs textes peuvent apporter des informations supplémentaires : Notes, _Curatorial’s Remark_s, Label Text (cartel d’une exposition), Historical Attributions (registres) et Texts Entries (extraits de textes publiés).

Champ Subject – mots-clés et descripteurs iconographiques : Les œuvres iconographiques sont décrites avec des descripteurs de différents niveaux selon le système proposé par Jane E. Falk dans son ouvrage Subject Authority List (San Francisco Museum of Modern Art, 1983). D’un niveau à l’autre, on passe d’une catégorie générique (ex : Portrait) à une sous-catégorie plus spécifique (ex : male), puis la personne est identifiée avec son nom au 3e ou 4e niveau, et avec divers attributs pour les niveaux subséquents. Afin d’établir des liens entre nos différentes collections, des attributs spécifiques réfèrent parfois à celles-ci (ex : Ethnology, aboriginal; fashion, costume, historical, victorian; art, painting; etc.). Chaque catégorie se termine avec un point-virgule pour en introduire une autre. Le Subject Authority List de Falk est ensuite adapté à la spécificité photographique de la collection Notman lorsque celle-ci commence à être cataloguée à partir de 1996. Ex : la catégorie Genre Scene est remplacée par la catégorie Occupation, et une catégorie d’attributs est ajoutée au Thésaurus : Portrait Format (full-length, half-length, head-and-shoulders, three-quarter-length, standing, sitting, etc.). Enfin, pour des représentations de personnes sur des objets de culture matérielle nous utilisons le thésaurus iconographique de Garnier.

Exemples :

1. Le portrait de Nicolas Vincent (proposé également par @marielmat, 2006-981)

[Christian-McCord]

2. Une photo par Jules Ernest Benoît dit Livernois:

[Christian-McCord]

3. Un portrait de groupe par Alexander Henderson:

[Christian-McCord]

4. Un tableau par Cornelius Krieghoff:

[Christian-McCord]

5. Un éventail de fabrication autochtone comprenant des ferrotypes

[Christian-McCord]

6. Une caricature par R. Pier publiée dans le Journal de Montréal en 1977

[stephenhart8]

Merci @Christian-McCord pour ces nombreux exemples!

I have quickly looked at them, and it helps to define what we need to describe actors within images.

There are three ways of documenting the subjects of Propositional Object (that contain texts, images, etc.). By a more general property P67_refers_to and to sub-properties P129_is_about and P138_represents.

• P67 refers to

  This property documents that an instance of E89 Propositional Object makes a statement about an instance of E1 CRM Entity. P67 refers to (is referred to by) has the P67.1 has type link to an instance of E55 Type. This is intended to allow a more detailed description of the type of reference. This differs from P129 is about (is subject of), which describes the primary subject or subjects of the instance of E89 Propositional Object.

• P129 is about

  This property documents that an instance of E89 Propositional Object has as subject an instance of E1 CRM Entity.

  This differs from P67 refers to (is referred to by), which refers to an instance of E1 CRM Entity, in that it describes the primary subject or subjects of an instance of E89 Propositional Object.

• P138 represents

  This property establishes the relationship between an instance of E36 Visual Item and the instance of E1 CRM Entity that it visually represents.

  Any entity may be represented visually. This property is part of the fully developed path from E24 Physical Human-Made Thing through P65 shows visual item (is shown by), E36 Visual Item, P138 represents (has representation) to E1 CRM Entity, which is shortcut by P62 depicts (is depicted by). P138.1 mode of representation allows the nature of the representation to be refined.

  This property is also used for the relationship between an original and a digitisation of the original by the use of techniques such as digital photography, flatbed or infrared scanning. Digitisation is here seen as a process with a mechanical, causal component rendering the spatial distribution of structural and optical properties of the original and does not necessarily include any visual similarity identifiable by human observation.

The property P67_refers_to is very general as it can link any propositional object to another entity it is mentioning, in any way possible. P129_is_about is a more precise property, is it can only be used to describe the main topic(s) of a propositional object, even if there can be multiple ones. Finally P138_represents is a property that directly links a visual item to something it refers to. It does not need to be the main subject.

In the pattern for image description, the only two properties needed are, in my opinion, P138 to identify any entity that is depicted, and P129 when the subject(s) of the visual item has(have) to be documented. What belongs to the subject of is merely an entity depicted is decided by the cataloguer.

In the case of example 6 (M2009.80.15), one option would be to have:

• "Cartoon" would be the mode of representation of René Lévesque and Pierre Elliott Trudeau (other modes could be added, as "sitting", etc.)
• "politics", "federal", "iberal", "provincial" could be the subject of the visual item
• "prime minister" could be inferred if, in the search, we are looking for a representation of an actor who has been prime minister in their occupation. The same would be for the term "male"
• "René Lévesque" and "Pierre Elliott Trudeau" would be represented on the visual item and therefore the actors directly linked to the image
• "Pair" is the most difficult term to represent. Either it is inferred by searching images with only 2 persons represented, or by adding the term "pair" to the subject of the image.

The distinction between the mode of representation of an actor and the subject or representation of concepts can vary depending on the cataloger. In the case of the example "moustache" of image M2017.46.2.4533 or "smoking" of M2000.95.1, it could either be possible to document the mode of representation of an actor (the person is represented with a moustache or drinking) but it could also be possible to document that those visual items represent the occupation of smoking (M2000.95.1) or has as one of its subjects "moustache" (M2017.46.2.4533).

We will be able to discuss more on the next meeting on the 4th of March

[stephenhart8]

From the discussions at the semantic committee on the 4th of March, a common agreement on the pattern for Images has been found.

• The distinction between two Visual Items is a decision the documenting party has to make. Links have been made in the model to document the relationship between different Visual Items
• The URL of the image can easily be documented in CIDOC CRM, it is therefore preferable to stay within the ontology and not use external ones (like schema.org)
• E41 and E42 can both be used to classify a URL, although it is more common to use E42. This is the choice of CHIN.
• It is better practice to have the URI of the E42 different from the URL of the image, even if, by using content management tools, it would be possible to use the URL as the URI of the E42.
• The description of an actor within an image is insufficient. Because this issue would require developing a pattern for image description, it has been decided to create a new issue for this topic: Issue #64 "Visual Item Description".
• The documentation of sources and rights, because it concerns not only visual items but other types of content (text for example), should be discussed in the appropriate issue: issue #38 "Should we document Rights Statements and Licenses, and how?"

The modifications of the pattern have been made in the TM 2.1, except for the description of images and sources, as it will be discussed and decided in other issues. Therefore, this issue is closed.