Support unknown concepts

[stefandesu]

I've had this idea when we recently talked about synthetic notations in DDC. We could allow displaying non-existent concepts by deconstructing the URI to get the notation. Of course, those concepts would have no other information, i.e. no label or additional information. If we then allow people to "create" new concepts by providing a notation, those new concepts could be used for mappings.

In case of DDC, we could at some point show the decomposition of the notation so that there would be some additional information about those concepts. For other concept schemes, we could try to determine possibly related concepts by slicing the notation.

[stefandesu]

If we have a regex for the format of notations for a certain concept scheme, we could limit "creation" of notations to valid notations only.

[nichtich]

JSKOS field notationPattern can hold the regex.

[nichtich]

See https://api.dante.gbv.de/konzept_ingest/ for DANTE API for adding concepts to an existing vocabulary. Leaving aside authentification, the payload is just a JSON array of JSKOS records with common inScheme field.

[stefandesu]

I think we should also differentiate between 1. using concepts with non-existing notations for mapping (for example built DDC numbers) and 2. actually creating new concepts (containing also prefLabel, hierarchy, etc.). 1. would not require writing to the database, while 2. would be more complicated because not only do they need to be saved in the database, but they have to be protected from deletion during a reimport.

[nichtich]

New/non-existing concepts should only be supported as part of mappings - if the mapping is removed, the new concept can be deleted as well.

For 1 (save the new concept in the mapping) there is a problem of duplicates: a new concept could only be used in one mapping or we cannot avoid duplicates. But if two users suggest the same new concept, we have a problem anyway - unless there are rules how to build new concepts (possible in DDC, not possible in GND).

For 2 the new concepts could be written to a dedicated registry. Merging this registry with the primary registry for the vocabulary might be difficult though.

Aynhow, a set of user stories might help to better the actual requirements of this feature.

[nichtich]

This feature was also requested at RVK user group for mapping DDC numbers. We should implement it as variant 1. using concepts with non-existing notations for mapping (for example built DDC numbers). The user can open a dialogue from mapping editor to enter a notation which URI is derived from (labels are not supported for this use case). Custom concepts must be enabled per concept scheme.

[stefandesu]

That sounds good. 👍 Also, the problem you mentioned earlier about duplicates shouldn't actually be a problem because if two people create the exact same concept (using the same notation), it just is the same concept, not a duplicate.

You suggested that the concepts should be created through the Mapping Editor. I understand the reasoning because you basically only need it for the mapping, but what do you think of my idea (which I apparently haven't noted down here) to use the search field for this. A user types in a notation, and either this notation already exists and the existing concept can be used, or if it doesn't exist and it is a valid notation for the current scheme, it will show up there as a "new concept" (with only a notation, no label). The user could then either select that and see if someone else has created mappings with that notation already (which is unlikely), or they could drag it directly from the search results into Mapping Editor. This would be much easier to implement and we wouldn't need an additional button anywhere.

[nichtich]

Fully agree with your idea of using the search box for creation of new concepts. In contrast to my previous comment I'd not limit this functionality to specific concept schemes but allow it for all concept schemes where we can derive URI from syntactically correct notation. This would also allow for mapping concept schemes which we don't have access to!

User-generated concepts and concept which we cannot lookup by URI should have a clear indicator in ConceptDetail, e.g. a warning icon and a message "No data available! It could be that this concept is not a valid member of {conceptScheme}."

[stefandesu]

User-generated concepts and concept which we cannot lookup by URI should have a clear indicator in ConceptDetail, e.g. a warning icon and a message "No data available! It could be that this concept is not a valid member of {conceptScheme}."

Good idea! I would put the icon in front of the concept (like the puzzle icon for built numbers) and show the message on hover. Let's implement this in 1.3.0. 👍

[stefandesu]

So I added a first basic implementation of this. Now, if an URI can be constructed from the search query as a notation, this will be added to the search results (if it's not already in the results). Right now, it just shows up with the notation only as if it was a result. I would add the indicator icon here as well.

Which icon should we use to indicate these concepts?

[nichtich]

Which icon should we use to indicate these concepts?

The best I could find was bolt icon, star icon, feather icon and skull icon 1 or skull icon 2. How about the bolt?

[stefandesu]

I like the bolt and the feather. I guess neither really represents what it really is, but for now it should be fine. I'm going to try both and see what looks better in practice.

[stefandesu]

I think the feather looks nicer, but the bolt feels like it makes more sense. I'm going to use the bolt, but there's a thing or two I need to figure out before I can commit.

[stefandesu]

Please try it out in dev and feel free to adjust the tooltip. Note that there is no tooltip on the entry in the search results because it did not render correctly. I would leave it for now, but I can look into it again.

This is a good base to get started with adding DDC number decomposition (is there an issue for that yet? I couldn't find one).

[nichtich]

I found a short tooltip is enough. In the search result it should be shown instead of the concept label, possibly in italics to differentiate from normal labels. I tried with DDC, RVK, and BK with success but at Wikidata the icon/tooltip is not shown (try Q7 or see Concordance Registry (dev)). Apart from that the feature is good as it is!

[stefandesu]

but at Wikidata the icon/tooltip is not shown (try Q7 or see Concordance Registry (dev))

This is a bug in wikidata-jskos: https://coli-conc.gbv.de/services/wikidata/concept/?uri=http:%2F%2Fwww.wikidata.org%2Fentity%2FQ7

[stefandesu]

I found a short tooltip is enough. In the search result it should be shown instead of the concept label, possibly in italics to differentiate from normal labels.

Done. 👍

[stefandesu]

If there are any other changes necessary, please reopen or create a new issue.