Modified and created dates are serialized as literals

dannylamb commented 7 years ago

When serializing an entity as jsonld, the modified and created dates are being written as literals with the sites default language. They should be serialized as instances of http://www.w3.org/2001/XMLSchema#dateTime instead.

To reproduce:

Create a new collection
Retrieve its jsonld representation. This can be done by appending ?_format=jsonld to the end of its URL and visiting/cURLing it. If cURLing, you'll need to provide an authentication token. If you're visiting it in the browser, you will have to enable cookie authentication through the restui interface at /admin/config/services/rest

You should see something like this:

{
"@graph":[
    {
        "@id":"http:\/\/localhost:8000\/node\/1?_format=jsonld",
        "@type":[
            "http:\/\/pcdm.org\/models#Collection",
            "http:\/\/schema.org\/CollectionPage"
        ],
        "http:\/\/purl.org\/dc\/terms\/title":[
            {
                "@value":"A Collection",
                "@language":"en"
            }
        ],
        "http:\/\/schema.org\/author":[
            {
                "@id":"http:\/\/localhost:8000\/user\/1?_format=jsonld"
            }
        ],
        "http:\/\/schema.org\/dateCreated":[
            {
                "@value":"2017-06-13T13:14:55+00:00",
                "@language":"en"
            }
        ],
        "http:\/\/schema.org\/dateModified":[
            {
                "@value":"2017-06-13T13:15:04+00:00",
                "@language":"en"
            }
        ],
        "http:\/\/purl.org\/dc\/terms\/description":[
            {
                "@value":"Holds stuff",
                "@language":"en"
            }
        ]
    },
    {
        "@id":"http:\/\/localhost:8000\/user\/1?_format=jsonld",
        "@type":[
            "http:\/\/schema.org\/Person"
        ]
    }
]
}

If you do a lookup in Gemini and track down the resource in Fedora, it'll be a little more obvious it's not a date when compared to the Fedora created and modified dates:

curl -H "Authorization: Bearer islandora" http://localhost:8000/gemini/drupal/node/1 Returned d9/51/33/3b/d951333b-8b7e-429c-92ee-d161af59e401. So you can look in up in Fedora with

curl -H "Authorization: Bearer islandora" http://localhost:8081/fcrepo/rest/d9/51/33/3b/d951333b-8b7e-429c-92ee-d161af59e401`

<http://localhost:8080/fcrepo/rest/d9/51/33/3b/d951333b-8b7e-429c-92ee-d161af59e401>
    rdf:type               pcdm:Collection ;
    rdf:type               fedora:Container ;
    rdf:type               schema:CollectionPage ;
    rdf:type               fedora:Resource ;
    fedora:lastModifiedBy  "bypassAdmin"^^<http://www.w3.org/2001/XMLSchema#string> ;
    schema:dateModified    "2017-06-13T13:15:04+00:00"@en ;
    schema:author          <http://localhost:8000/user/1?_format=jsonld> ;
    schema:dateCreated     "2017-06-13T13:14:55+00:00"@en ;
    fedora:createdBy       "bypassAdmin"^^<http://www.w3.org/2001/XMLSchema#string> ;
    fedora:lastModified    "2017-06-13T13:15:09.541Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> ;
    fedora:created         "2017-06-13T13:15:09.541Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> ;
    dcterms:description    "Holds stuff"@en ;
    dcterms:title          "A Collection"@en ;
    rdf:type               ldp:RDFSource ;
    rdf:type               ldp:Container ;
    fedora:writable        "true"^^<http://www.w3.org/2001/XMLSchema#boolean> ;
    fedora:hasParent       <http://localhost:8080/fcrepo/rest/> .

The scope of this ticket is to modify the https://github.com/Islandora-CLAW/jsonld module to properly serialize dates. It can be tested by performing the same steps as above, and the date should not be given a language and it should appear as a http://www.w3.org/2001/XMLSchema#dateTime in Fedora.

kimpham54 commented 7 years ago

hi @dannylamb, If I wanted to test this ticket/work on it is the issue happening on the 8.x-1.x branch or is it on the commit that the CLAW repo uses, https://github.com/Islandora-CLAW/jsonld/tree/eeec5aada39480615aa328eff2b0d5390dd75309? Which branch should I work off of?

whikloj commented 7 years ago

@kimpham54 you should work off of 8.x-1.x. But so you are aware @ajs6f was looking at this too and I thought that @DiegoPino had done some work on this. So he may have a simple solution up his sleeve.

dannylamb commented 7 years ago

@kimpham54 Eventually, the commit hashes that the CLAW repo uses will be for releases. So if you wanted to work off a commit tied to particular release version, that'd be the way to go. But mainline development should go off of the 8.x-1.x branch as @whikloj suggested.

FWIW we're not keeping up on updating the CLAW repo very well. Ideally it'd be some sort of automated process, but for now, I wouldn't trust them.

ajs6f commented 7 years ago

Yes, @DiegoPino definitely has some ideas here and should be pinged before work goes forward.

kimpham54 commented 7 years ago

Thanks all for the information. I'll still try to take a look out of interest

whikloj commented 7 years ago

For the reference of whomever might take this on. I noted that if you alter the rdf mapping you can add a datatype: property which is added to the field in the JSON-LD as @type.

See Islandora IRC logs 2017-06-15 starting at 09:52

DiegoPino commented 7 years ago

@whikloj @ajs6f wonder about the @en part now. Also, i will fix some data types in absence of explicit typing as i did here: https://github.com/Islandora-CLAW/jsonld/blob/8.x-1.x/src/ContextGenerator/JsonldContextGenerator.php#L321-L422 Will have a pull for tomorrow.

DiegoPino commented 7 years ago

@whikloj @ajs6f since i'm here and have a fix in hand, wonder if there are other miss typed properties you have seen around before making the pull request?

whikloj commented 7 years ago

@DiegoPino just wondering if you are still in here. I was doing some digging around and using the getFieldsRdf inside FieldItemNormalizer (right after this }) I was able to change this:

[
  {
    "@id": "http://localhost:8000/node/1?_format=jsonld",
    "@type": [
      "http://pcdm.org/models#Object",
      "http://schema.org/ImageObject"
    ],
    "http://pcdm.org/models#hasFile": [
      {
        "@id": "http://localhost:8000/media/1?_format=jsonld"
      }
    ],
    "http://purl.org/dc/terms/description": [
      {
        "@language": "en",
        "@value": "This is some content"
      }
    ],
    "http://purl.org/dc/terms/title": [
      {
        "@language": "en",
        "@value": "Test Image"
      }
    ],
    "http://schema.org/author": [
      {
        "@id": "http://localhost:8000/user/1?_format=jsonld"
      }
    ],
    "http://schema.org/dateCreated": [
      {
        "@language": "en",
        "@value": "2017-07-12T13:43:27+00:00"
      }
    ],
    "http://schema.org/dateModified": [
      {
        "@language": "en",
        "@value": "2017-07-12T13:44:16+00:00"
      }
    ]
  },
  {
    "@id": "http://localhost:8000/user/1?_format=jsonld",
    "@type": [
      "http://schema.org/Person"
    ]
  },
  {
    "@id": "http://localhost:8000/media/1?_format=jsonld",
    "@type": [
      "http://pcdm.org/use#ServiceFile"
    ]
  }
]

to this

[
  {
    "@id": "http://localhost:8000/node/1?_format=jsonld",
    "@type": [
      "http://pcdm.org/models#Object",
      "http://schema.org/ImageObject"
    ],
    "http://pcdm.org/models#hasFile": [
      {
        "@id": "http://localhost:8000/media/1?_format=jsonld"
      }
    ],
    "http://purl.org/dc/terms/description": [
      {
        "@type": "xsd:string",
        "@value": "This is some content"
      }
    ],
    "http://purl.org/dc/terms/title": [
      {
        "@type": "xsd:string",
        "@value": "Test Image"
      }
    ],
    "http://schema.org/author": [
      {
        "@id": "http://localhost:8000/user/1?_format=jsonld"
      }
    ],
    "http://schema.org/dateCreated": [
      {
        "@type": "xsd:dateTime",
        "@value": "2017-07-12T13:43:27+00:00"
      }
    ],
    "http://schema.org/dateModified": [
      {
        "@type": "xsd:dateTime",
        "@value": "2017-07-12T13:44:16+00:00"
      }
    ]
  },
  {
    "@id": "http://localhost:8000/user/1?_format=jsonld",
    "@type": [
      "http://schema.org/Person"
    ]
  },
  {
    "@id": "http://localhost:8000/media/1?_format=jsonld",
    "@type": [
      "http://pcdm.org/use#ServiceFile"
    ]
  }
]

I had to stop adding the @language property if there was an @type one, apparently JSON-LD doesn't like that.

Still need to expand the namespaces, but I wanted to see how you were doing. If you're close then I'll move on, but if you have more tasks than time I might be able to figure this out.

DiegoPino commented 7 years ago

@whikloj go for it, no worries. I had this open question

wonder if there are other miss typed properties you have seen around before making the pull request?

and then i forgot about it because i'm a flawed, over-committed, and flawed (again) human being. Do you plan to remove language at all?

DiegoPino commented 7 years ago

And @whikloj yep, my approach was exactly that, use the context generator code, so cool

whikloj commented 7 years ago

@DiegoPino I had left @language in there, but the json-ld.org/playground threw an error when parsing the resulting output because I had both @type and @language.

In the Spec I read this:

Language associations are only applied to plain strings. Typed values or values that are subject to type coercion are not language tagged.

So I guess we could leave @language if there is no @type because we would assume it is a string literal. What do you think?

ajs6f commented 7 years ago

That "no language for a non-string-typed literal" rule comes right out of RDF, so it's even more general than JSON-LD.

dannylamb commented 7 years ago

Resolved via https://github.com/Islandora-CLAW/jsonld/commit/36f37cb2bae7467fd6d3eacabd9e4a368ea77a7f

Islandora / documentation

Modified and created dates are serialized as literals #661