search

hbz / lobid-resources

Transformation, web frontend, and API for the hbz catalog as LOD
http://lobid.org/resources
Eclipse Public License 2.0
8 stars 7 forks source link

Ordered lists in JSON-LD #4

Closed dr0i closed 7 years ago

dr0i commented 9 years ago

@acka47 wrote in #1:

Unfortunately, the arrays for creators/contributors and subjects aren't sorted according to the source data as we would like them to. For this, we will need to

  1. add the order in N-Triples as RDF list or
  2. discard this approach and move on like we intended to first, i.e. directly creating JSON-LD from source data with Metafacture.
dr0i commented 9 years ago

What is the exact structure we need? Something like this is possible with the @listcontainer in context (see specs):

"@id": "http://example.org/people#joebob",
"nick": [ "joe", "bob", "jaybee" ],

If I got the json-ld specs right:

List of lists in the form of list objects are not allowed in this version of JSON-LD. This decision was made due to the extreme amount of added complexity when processing lists of lists. (...)A list object MUST be a JSON object that contains no keys that expand to an absolute IRI or keyword other than @list, @context, and @index.

it's not possible to do the following:

"@id": "http://example.org/people#joebob",
"jobTitle": [ 
                  { 
                   "@id": "http://gnd.de/12345",
                   "preferredName": "Informatiker"
                  }, {
                   "@id": "http://gnd.de/54321",
                   "preferredName": "Bibleothekar"
                  }
              ],
jschnasse commented 9 years ago

Third possible approach is to create your JSON-LD from rdf, but manually. You can type fields like "jobTitle" simply as @id. This works well with java json-ld so far. Ok, if converted back to rdf the array ist not renderer as list, so in that case the ordering will be lost. But in your current setup the correct rdf is already there so why care about? Json ordering is given by the array type. Rdf ordering ist given by the rdf-list. Mission accomplished.

acka47 commented 9 years ago

I created a simplified example.

JSON-LD file

{
  "@graph" : [ {
    "@id" : "http://d-nb.info/gnd/1060772442",
    "preferredName" : "Pfarrei St. Joseph und St. Antonius Bergisch Gladbach",
    "preferredNameForTheCorporateBody" : "Pfarrei St. Joseph und St. Antonius Bergisch Gladbach",
    "preferredNameForThePerson" : "Pfarrei St. Joseph und St. Antonius Bergisch Gladbach"
  }, {
    "@id" : "http://d-nb.info/gnd/172845785",
    "preferredName" : "Clemens-Schierbaum, Ursula, 1963-",
    "preferredNameForThePerson" : "Clemens-Schierbaum, Ursula, 1963-"
  }, {
    "@id" : "http://d-nb.info/gnd/2175350-7",
    "preferredName" : "Pfarrei Sankt Antonius Abbas Herkenrath",
    "preferredNameForTheCorporateBody" : "Pfarrei Sankt Antonius Abbas Herkenrath",
    "preferredNameForThePerson" : "Pfarrei Sankt Antonius Abbas Herkenrath"
  }, {
    "@id" : "http://lobid.org/resource/HT018454638",
    "@type" : [ "http://purl.org/dc/terms/BibliographicResource", "http://purl.org/ontology/bibo/Book" ],
    "contributor" : [ "http://d-nb.info/gnd/1060772442", "http://d-nb.info/gnd/2175350-7", "http://d-nb.info/gnd/172845785" ],
    "language" : "http://id.loc.gov/vocabulary/iso639-2/deu",
    "medium" : "http://rdvocab.info/termList/RDAproductionMethod/1010",
    "subject" : [ "http://d-nb.info/gnd/2175350-7", "http://d-nb.info/gnd/1060772442" ],
    "exemplar" : [ "http://lobid.org/item/HT018454638:DE-Kn28:Fc%206601", "http://lobid.org/item/HT018454638:DE-5-39:Gb%207470%2F201" ],
        "describedBy" : "http://lobid.org/resource/HT018454638/about"
  } ],
  "@id" : "http://lobid.org/resource/HT018454638/about",
  "dateCreated" : "20141111",
  "dateModified" : "20150303",
  "@context" : "http://lobid.org/context/lobid-resources.json"
}

Frame

:exclamation: Note that I am using @list as well as `@set in the example.

{
    "@type": "http://purl.org/dc/terms/BibliographicResource",
    "@context": {
        "preferredName": "http://d-nb.info/standards/elementset/gnd#preferredName",
        "preferredNameForTheCorporateBody": "http://d-nb.info/standards/elementset/gnd#preferredNameForTheCorporateBody",
        "preferredNameForThePerson": "http://d-nb.info/standards/elementset/gnd#preferredNameForThePerson",
        "contributor": {
            "@type": "@id",
            "@id": "http://purl.org/dc/terms/contributor",
            "@container": "@list"
            },
        "language": {
            "@id": "http://purl.org/dc/terms/language",
            "@type": "@id",
            "@container": "@set"
            },
        "medium": {
            "@type": "@id",
            "@id": "http://purl.org/dc/terms/medium",
            "@container": "@set"
            },
    "subject": {
        "@id": "http://purl.org/dc/terms/subject",
        "@type": "@id",
        "@container": "@list"
        },
    "exemplar": {
        "@type": "@id",
        "@id": "http://purl.org/vocab/frbr/core#exemplar",
        "@container": "@set"
        },
    "dateCreated": "http://purl.org/dc/terms/created",
    "dateModified": "http://purl.org/dc/terms/modified",
    "describedBy": {
        "@type": "@id",
        "@id": "http://www.w3.org/2007/05/powder-s#describedby"
        }
    }
}

Playground

Using both in the playground doesn't give an error but strangely the keys for a @list that is an array of objects become URIs again. See http://tinyurl.com/qzfp588.

acka47 commented 9 years ago

I guess this just doesn't work and @dr0i interpreted this notion from the spec correctly:

List of lists in the form of list objects are not allowed in this version of JSON-LD. This decision was made due to the extreme amount of added complexity when processing lists of lists.

I learned one thing, though. You can use @set in the context to make all values of a specific property an array. This is what customers asked for in another contexct. You won't have order though...

acka47 commented 9 years ago

We will have to talk to @jschnasse tomorrow about his approach. Sounds appealing to me.

acka47 commented 9 years ago

Discussing this offline we decided to use this elegant solution to provide information on order with an extra triple. Example for expected outcome (snippet):

{
   "@context" : {
      "subject" : {
         "@id" : "http://purl.org/dc/terms/subject",
         "@type" : "@id"
      },
      "subjectOrder" : {
         "@id" : "http://purl.org/lobid/lv#subjectOrder",
         "@container" : "@list"
      },
      "exemplar" : {
         "@type" : "@id",
         "@id" : "http://purl.org/vocab/frbr/core#exemplar"
      },
      "dateModified" : "http://purl.org/dc/terms/modified",
      "contributor" : {
         "@type" : "@id",
         "@id" : "http://purl.org/dc/terms/contributor"
      },
      "contributorOrder" : {
         "@container" : "@list",
         "@id" : "http://purl.org/lobid/lv#contributorOrder"
      },
      "preferredName" : "http://d-nb.info/standards/elementset/gnd#preferredName",
      "language" : {
         "@type" : "@id",
         "@id" : "http://purl.org/dc/terms/language"
      },
      "preferredNameForThePerson" : "http://d-nb.info/standards/elementset/gnd#preferredNameForThePerson",
      "describedBy" : {
         "@type" : "@id",
         "@id" : "http://www.w3.org/2007/05/powder-s#describedby"
      },
      "dateCreated" : "http://purl.org/dc/terms/created",
      "preferredNameForTheCorporateBody" : "http://d-nb.info/standards/elementset/gnd#preferredNameForTheCorporateBody",
      "medium" : {
         "@type" : "@id",
         "@id" : "http://purl.org/dc/terms/medium"
      }
   },
   "@id" : "http://lobid.org/resource/HT018454638",
   "medium" : "http://rdvocab.info/termList/RDAproductionMethod/1010",
   "language" : "http://id.loc.gov/vocabulary/iso639-2/deu",
   "contributor" : [
      {
         "preferredNameForThePerson" : "Clemens-Schierbaum, Ursula, 1963-",
         "@id" : "http://d-nb.info/gnd/172845785",
         "preferredName" : "Clemens-Schierbaum, Ursula, 1963-"
      },
      {
         "preferredNameForThePerson" : "Pfarrei Sankt Antonius Abbas Herkenrath",
         "@id" : "http://d-nb.info/gnd/2175350-7",
         "preferredNameForTheCorporateBody" : "Pfarrei Sankt Antonius Abbas Herkenrath",
         "preferredName" : "Pfarrei Sankt Antonius Abbas Herkenrath"
      },
      {
         "preferredName" : "Pfarrei St. Joseph und St. Antonius Bergisch Gladbach",
         "preferredNameForTheCorporateBody" : "Pfarrei St. Joseph und St. Antonius Bergisch Gladbach",
         "@id" : "http://d-nb.info/gnd/1060772442",
         "preferredNameForThePerson" : "Pfarrei St. Joseph und St. Antonius Bergisch Gladbach"
      }
   ],
   "contributorOrder" : [
      "http://d-nb.info/gnd/172845785",
      "http://d-nb.info/gnd/1060772442",
      "http://d-nb.info/gnd/2175350-7"
   ],
   "@type" : [
      "http://purl.org/dc/terms/BibliographicResource",
      "http://purl.org/ontology/bibo/Book"
   ],
   "subject" : [
      {
         "preferredName" : "Pfarrei Sankt Antonius Abbas Herkenrath",
         "preferredNameForTheCorporateBody" : "Pfarrei Sankt Antonius Abbas Herkenrath",
         "@id" : "http://d-nb.info/gnd/2175350-7",
         "preferredNameForThePerson" : "Pfarrei Sankt Antonius Abbas Herkenrath"
      },
      {
         "preferredNameForTheCorporateBody" : "Pfarrei St. Joseph und St. Antonius Bergisch Gladbach",
         "@id" : "http://d-nb.info/gnd/1060772442",
         "preferredName" : "Pfarrei St. Joseph und St. Antonius Bergisch Gladbach",
         "preferredNameForThePerson" : "Pfarrei St. Joseph und St. Antonius Bergisch Gladbach"
      }
   ],
   "subjectOrder" : [
      "http://d-nb.info/gnd/1060772442",
      "http://d-nb.info/gnd/2175350-7",
      "Geschichte"
   ],
   "exemplar" : [
      "http://lobid.org/item/HT018454638:DE-Kn28:Fc%206601",
      "http://lobid.org/item/HT018454638:DE-5-39:Gb%207470%2F201"
   ],
   "describedBy" : {
      "dateCreated" : "20141111",
      "@id" : "http://lobid.org/resource/HT018454638/about",
      "dateModified" : "20150303"
   }
}

This approach has at least the benefits (that's why I wouldn't call it a workaround anymore):

I guess, with this decision, we can close this issue and move on at #1 .

fsteeg commented 9 years ago

To add one point: I don't think our conclusion was that we can avoid rdf:Lists altogether, as we still need them in the n-triples to generate the ordered arrays in your example above. But we avoid them in the JSON-LD.

dr0i commented 9 years ago

But is "@container" : "@list" not a list in JSON-LD? Thus, we still have rdf:lists under the hood. We avoid lists in lists though, which is nice :+1: .

acka47 commented 9 years ago

But is "@container" : "@list" not a list in JSON-LD? Thus, we still have rdf:lists under the hood.

You are right. We don't really avoid them, neither in the transformation process nor in the resulting data. I adjusted my comment accordingly. (We could avoid rdf:Lists altogether, though, by putting information on order in just one string but you coders decided that a list would be better.)

dr0i commented 9 years ago

Filed https://github.com/jsonld-java/jsonld-java/issues/150 as the embedding in frames is not what we expect.

acka47 commented 9 years ago

Just looked at some and was reminded of bibo:authorList and bibo:contributorList. Before finishing this one, we should think about using these as they have the appropriate rdfs:range of rdf:List and rdf:Seq.

dr0i commented 9 years ago

@acka47 please have a look at hbz/lobid-rdf-to-json#4 , especially to the file hbz01.es.json where I added @id to the exemplar (This file is the template for the output we want to have) to be in accordance to the JsonConverter (which adds the @id when converting ntriples to json-ld).

acka47 commented 9 years ago

I took a look at hbz01.es.json. We don't need the @id bit as the type is already defined in the lobid-resources json-ld context (snippet):

{
"exemplar": {
  "@type": "@id",
  "@id": "http://purl.org/vocab/frbr/core#exemplar"
  }
}

BTW, the JSON-LD context is not yet linked to in the template file.

dr0i commented 9 years ago

If by saying

We don't need the @id bit as the type is already defined in the lobid-resources json-ld context (snippet):

you mean that's ok to have @id in the data at that position, ignore this comment. I doubt that though and guess that what you meant was rather it musn't be there - then please open a new issue for that.

BTW, the JSON-LD context is not yet linked to in the template file.

Yes,but it wouldn't help us anyway because we don't use a JSON-LD library but only json-library (which cannot handle @context the way it's meant in LD context).

dr0i commented 9 years ago

Musing about having more than one subject chain (like e.g. in http://lobid.org/resource/BT000128754/about ): wouldn't that mean to have ordered lists in ordered lists? Or does the sequentiality of subjectOrder not matter? (Btw., I couldn't lookup the property http://purl.org/lobid/lv#subjectOrdered so I haven't a hint about it's semantics).

acka47 commented 9 years ago

Musing about having more than one subject chain (like e.g. in http://lobid.org/resource/BT000128754/about ): wouldn't that mean to have ordered lists in ordered lists? Or does the sequentiality of subjectOrder not matter?

No we don't need to retain the order of various subject chains. Thus, the sequentiality of subjectOrder does not matter.

(Btw., I couldn't lookup the property http://purl.org/lobid/lv#subjectOrdered so I haven't a hint about it's semantics).

I haven't added it yet as we don't use it in productiuon. BTW, it shoudl read http://purl.org/lobid/lv#subjectOrder

dr0i commented 8 years ago

Deployed to staging, see http://gaia.hbz-nrw.de:9200/resources/resource/HT018454638.

acka47 commented 8 years ago

+1

acka47 commented 8 years ago

it's not possible to do the following:

 "@id": "http://example.org/people#joebob",
    "jobTitle": [ 
                      { 
                       "@id": "http://gnd.de/12345",
                       "preferredName": "Informatiker"
                      }, {
                       "@id": "http://gnd.de/54321",
                       "preferredName": "Bibleothekar"
                      }
                  ],

I reopened this as the following actually is possible which was shown to me by @jschnasse :

{
  "@context": {
    "jobTitle": {
      "@id": "http://example.org/jobTitle",
      "@container": "@list"
        },
    "preferredName": "http://example.org/preferredName"
  },
  "@id": "http://example.org/people#joebob",
  "jobTitle": [ 
                  { 
                   "@id": "http://gnd.de/12345",
                   "preferredName": "Informatiker"
                  }, {
                   "@id": "http://gnd.de/54321",
                   "preferredName": "Bibliothekar"
                  }
              ]
}

See http://tinyurl.com/jq6x7ut for proof. This means that the whole discussion and solution was based on a false assumption. :smirk:

acka47 commented 8 years ago

As lists of objects are possible with JSON-LD and we group all contributors together as objects in the "contributor" array using "role" for indicatin the relator code (https://github.com/hbz/lobid-resources/issues/38), we can go ahead and remove the "contributorOrder" and "subjectOrder" statements and instead create the following structure:

{
   "@context" : {
      "id": "@id",
      "type": "@type",
      "subject" : {
         "@id" : "http://purl.org/dc/terms/subject",
         "@type" : "@id",
         "@container": "@list"
      },
      "contributor" : {
         "@type" : "@id",
         "@id" : "http://purl.org/dc/terms/contributor",
         "@container": "@list"
      },
      "label": "http://www.w3.org/2000/01/rdf-schema#label",
      "altLabel": "http://www.w3.org/2004/02/skos/core#altLabel",
      "BibliographicResource": "http://purl.org/dc/terms/BibliographicResource",
      "Book": "http://purl.org/ontology/bibo/Book",
      "CorporateBody": "http://d-nb.info/standards/elementset/gnd#CorporateBody",
      "DifferentiatedPerson": "http://d-nb.info/standards/elementset/gnd#DifferentiatedPerson"
   },
   "@id" : "http://lobid.org/resources/HT018454638#!",
   "contributor" : [
      {
         "id" : "http://d-nb.info/gnd/172845785",
         "type": "DifferentiatedPerson",
         "label" : "Clemens-Schierbaum, Ursula",
         "altLabel": [ "Clemens, Ursula", "Schierbaum, Ursula C.-", "Schierbaum, Ursula Clemens-" ]
      },
      {
         "@id" : "http://d-nb.info/gnd/1060772442",
         "type": "CorporateBody",
         "label" : "Pfarrei St. Joseph und St. Antonius Bergisch Gladbach",
         "altLabel": ["Katholische Kirchengemeinde St. Joseph und St. Antonius Bergisch Gladbach", "Pfarrei Sankt Joseph und Sankt Antonius Bergisch Gladbach" ]
      },
      {
         "@id" : "http://d-nb.info/gnd/2175350-7",
         "type": "CorporateBody",
         "label" : "Pfarrei Sankt Antonius Abbas Herkenrath",
         "altLabel": [ "Pfarrei St. Antonius Abbas Herkenrath", "Pfarrei Sankt Antonius Abbas Bergisch Gladbach- Herkenrath", "Pfarrgemeinderat Sankt Antonius Abbas Herkenrath", "Katholische Kirchengemeinde Sankt Joseph und Sankt Antonius Bergisch Gladbach", "Pfarrei Sankt Antonius Abbas Herkenrath. Pfarrgemeinderat", "Pfarrgemeinderat. Pfarrei Sankt Antonius Abbas Herkenrath" ]
      }
   ],
   "type" : [
      "BibliographicResource",
      "Book"
   ],
   "subject" : [
        {
            "@id" : "http://d-nb.info/gnd/1060772442",
            "type": "CorporateBody",
            "label" : "Pfarrei St. Joseph und St. Antonius Bergisch Gladbach",
            "altLabel": ["Katholische Kirchengemeinde St. Joseph und St. Antonius Bergisch Gladbach", "Pfarrei Sankt Joseph und Sankt Antonius Bergisch Gladbach" ]
         },
         {
            "@id" : "http://d-nb.info/gnd/2175350-7",
            "type": "CorporateBody",
            "label" : "Pfarrei Sankt Antonius Abbas Herkenrath",
            "altLabel": [ "Pfarrei St. Antonius Abbas Herkenrath", "Pfarrei Sankt Antonius Abbas Bergisch Gladbach- Herkenrath", "Pfarrgemeinderat Sankt Antonius Abbas Herkenrath", "Katholische Kirchengemeinde Sankt Joseph und Sankt Antonius Bergisch Gladbach", "Pfarrei Sankt Antonius Abbas Herkenrath. Pfarrgemeinderat", "Pfarrgemeinderat. Pfarrei Sankt Antonius Abbas Herkenrath" ]
         },
        {
          "label": "Geschichte"
        }
   ]
}

The important thing is the right order in the subject and contributor arrays and the `"@container": "@list" statement in the context for "subject" and "contributor".

dr0i commented 8 years ago
dr0i commented 7 years ago

Deployed to staging, see http://test.lobid.org/resources/HT018454638.

acka47 commented 7 years ago

+1

dr0i commented 7 years ago

@fsteeg @jschnasse : please Ack if you are good with this. In particular, the contributorOrder field is removed since the contribution list is now ordered.

fsteeg commented 7 years ago

+1

dr0i commented 7 years ago

@jschnasse told me that he depends on contributorOrder for quite some time. So I will revert commit https://github.com/hbz/lobid-resources/commit/bd125565e87d81cbb3cdbb6ba6811430aff488a6.

dr0i commented 7 years ago

contributorOder exists again. Deployed to production. Closing.