search

w3c / wot-thing-description

Web of Things (WoT) Thing Description
http://w3c.github.io/wot-thing-description/
Other
131 stars 63 forks source link

TD 1.1 JSON-LD/RDF roundtripping example #2031

Open DavideRossi opened 1 month ago

DavideRossi commented 1 month ago

Can anyone provide a working example of TD roundtripping? Say we start with this simple example TD Link to thingweb TD playground Encoding to RDF looks simple, it can be done online using the JSON-LD Playground. So far so good, we have N-Quads RDF and we can use any RDF tool. Now I start having issues. How should I convert back from RDF to TD? Via compacting? Via framing (and where is the TD frame definition?) Can anyone provide a working toolchain? Online converters, code, whatever. I would just like to see this thing working.

egekorkan commented 1 month ago

@wiresio has some experience within https://github.com/eclipse-thingweb/domus-tdd-api . Could you help here?

wiresio commented 1 month ago

Hi @DavideRossi, as @egekorkan wrote you can find everything in the repo above, especially in: https://github.com/eclipse-thingweb/domus-tdd-api/blob/main/tdd/td.py and https://github.com/eclipse-thingweb/domus-tdd-api/blob/main/tdd/sparql.py

  1. JSON-LD to RDF
  2. Store RDF for each TD as a named graph
  3. Retrieve named graph via SPARQL construct query
  4. Convert N-Quads to JSON-LD
  5. Frame JSON-LD

When you extract the corresponding pieces from our code, you should be able to roundtrip your example via JSON-LD playground (JSON-LD to RDF), local Jena installation (store RDF and carry out the SPARQL query with JSON-LD as output format), and JSON-LD playground again (framing).

wiresio commented 1 month ago

In our AID plugin, available at https://github.com/wiresio/domus-tdd-api-plugin-aid, we go one step further and carry out a format conversion:

  1. TD JSON-LD to RDF
  2. Store TD RDF
  3. Retrieve via a different SPARQL construct query (TD -> AID)
  4. Convert N-Quads
  5. Frame with AAS context and retrieve AID JSON-LD
DavideRossi commented 1 month ago

Thanks a lot @wiresio I'm definitively going to look into that!

DavideRossi commented 1 month ago

I was able to single out the problem I'm having with roundtripping (I have some half-baked Kotlin code for a Thing Directory). Titanium JSON-LD 1.1 Processor complains about basic_sc not being an URI even with URI validation disabled. This does not apply to jsonld.js used by @wiresio, which is just happy with it. But it looks like Titanium is right, basic_sc is not an URI at all (IRIs, at a minimum, must have a ':' somewhere) and subjects in RDF (if memory serves me well) must be IRIs or blank nodes. So the question is: is this a bug in Titanium, is the TD I started with wrong (not well formed), is it an error in the TD specs (stating explicitly that TD can be non-well formed), or is there something else I'm missing?

wiresio commented 1 month ago

Please see here: https://github.com/w3c/wot-thing-description/issues/1193

DavideRossi commented 1 month ago

Thanks @wiresio , but I don't know if that is the same problem. If I got it right (and this may very well not be the case) I should replace the TD's @context with the latest td-context-1.1.jsonld in the repo. But still basic_sc is not an IRI. A solution could be adding a @vocab to the context (arguably using the TD's URL in the directory) but to me it looks like I'm just fiddling with it. Which is the correct way of dealing with that?

Post scriptum: I realized the @vocab solution cannot work, it will transform basic_sc to an IRI when used as a subject but, obviously, it will be left untouched when used as an object so that's clearly a no-go.

DavideRossi commented 1 month ago

OK, my previous message was clearly me misunderstanding what's going on here. Correct me if I'm wrong: the new td-context-1.1.jsonld introduces a new property to fix the security-related round trip problems. Which means that replacing the original TD's @context with this new one I should get a correct conversion to RDF. Well I did it, and still Titanium complains about basic_sc being not well formed. What am I missing this time? Am I supposed to further change something in the TD?

Notice, it's not just Titanium. I tried the RDF conversion with the JSON-LD Playground as well and the result is:

<urn:uuid:0804d572-cce8-422a-bb7c-4412fcd56f06> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://www.w3.org/2019/wot/td#Thing> .
<urn:uuid:0804d572-cce8-422a-bb7c-4412fcd56f06> <https://www.w3.org/2019/wot/td#definesSecurityScheme> _:b0 .
<urn:uuid:0804d572-cce8-422a-bb7c-4412fcd56f06> <https://www.w3.org/2019/wot/td#description> "Thing Description for a Lamp thing"@en .
<urn:uuid:0804d572-cce8-422a-bb7c-4412fcd56f06> <https://www.w3.org/2019/wot/td#hasActionAffordance> _:b1 .
<urn:uuid:0804d572-cce8-422a-bb7c-4412fcd56f06> <https://www.w3.org/2019/wot/td#hasEventAffordance> _:b3 .
<urn:uuid:0804d572-cce8-422a-bb7c-4412fcd56f06> <https://www.w3.org/2019/wot/td#hasPropertyAffordance> _:b6 .
<urn:uuid:0804d572-cce8-422a-bb7c-4412fcd56f06> <https://www.w3.org/2019/wot/td#hasSecurityConfiguration> "basic_sc" .
<urn:uuid:0804d572-cce8-422a-bb7c-4412fcd56f06> <https://www.w3.org/2019/wot/td#title> "MyLampThing"@en .
_:b0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://www.w3.org/2019/wot/security#BasicSecurityScheme> .
_:b0 <https://www.w3.org/2019/wot/security#in> "header" .
_:b1 <https://www.w3.org/2019/wot/td#hasForm> _:b2 .
_:b1 <https://www.w3.org/2019/wot/td#isIdempotent> "false"^^<http://www.w3.org/2001/XMLSchema#boolean> .
_:b1 <https://www.w3.org/2019/wot/td#isSafe> "false"^^<http://www.w3.org/2001/XMLSchema#boolean> .
_:b1 <https://www.w3.org/2019/wot/td#name> "toggle" .
_:b2 <https://www.w3.org/2019/wot/hypermedia#forContentType> "text/plain" .
_:b2 <https://www.w3.org/2019/wot/hypermedia#hasOperationType> <https://www.w3.org/2019/wot/td#invokeAction> .
_:b2 <https://www.w3.org/2019/wot/hypermedia#hasTarget> "https://mylamp.example.com/toggle"^^<http://www.w3.org/2001/XMLSchema#anyURI> .
_:b3 <https://www.w3.org/2019/wot/td#hasForm> _:b4 .
_:b3 <https://www.w3.org/2019/wot/td#hasNotificationSchema> _:b5 .
_:b3 <https://www.w3.org/2019/wot/td#name> "overheating" .
_:b4 <https://www.w3.org/2019/wot/hypermedia#forContentType> "text/plain" .
_:b4 <https://www.w3.org/2019/wot/hypermedia#forSubProtocol> "longpoll" .
_:b4 <https://www.w3.org/2019/wot/hypermedia#hasOperationType> <https://www.w3.org/2019/wot/td#subscribeEvent> .
_:b4 <https://www.w3.org/2019/wot/hypermedia#hasTarget> "https://mylamp.example.com/oh"^^<http://www.w3.org/2001/XMLSchema#anyURI> .
_:b5 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://www.w3.org/2019/wot/json-schema#StringSchema> .
_:b5 <https://www.w3.org/2019/wot/json-schema#readOnly> "true"^^<http://www.w3.org/2001/XMLSchema#boolean> .
_:b5 <https://www.w3.org/2019/wot/json-schema#writeOnly> "false"^^<http://www.w3.org/2001/XMLSchema#boolean> .
_:b6 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://www.w3.org/2019/wot/json-schema#StringSchema> .
_:b6 <https://www.w3.org/2019/wot/json-schema#readOnly> "false"^^<http://www.w3.org/2001/XMLSchema#boolean> .
_:b6 <https://www.w3.org/2019/wot/json-schema#writeOnly> "false"^^<http://www.w3.org/2001/XMLSchema#boolean> .
_:b6 <https://www.w3.org/2019/wot/td#hasForm> _:b7 .
_:b6 <https://www.w3.org/2019/wot/td#isObservable> "false"^^<http://www.w3.org/2001/XMLSchema#boolean> .
_:b6 <https://www.w3.org/2019/wot/td#name> "status" .
_:b7 <https://www.w3.org/2019/wot/hypermedia#forContentType> "text/plain" .
_:b7 <https://www.w3.org/2019/wot/hypermedia#hasOperationType> <https://www.w3.org/2019/wot/td#readProperty> .
_:b7 <https://www.w3.org/2019/wot/hypermedia#hasTarget> "https://mylamp.example.com/status"^^<http://www.w3.org/2001/XMLSchema#anyURI> .

where you have a id-less BasicSecurityScheme (_:b0) while the thing references "basic_sc" as its hasSecurityConfiguration...

wiresio commented 1 month ago

What happens if you try:

DavideRossi commented 3 weeks ago

First of, @wiresio, thanks a lot for you continued assistance. By trying to follow your advice I mixed all the possible combinations of 3 contexts: the one published at https://www.w3.org/2022/wot/td/v1.1, the latest version from the repo, and the one you're using in domus-ttd. I tried performing all the various conversion steps using both online tools (the JSON-LD Playground and the RDF Distiller) and Java code with Titanium, The issue with basic_sc not being an IRI is easily solved by adding a @base entry in the context (which I see you're doing in domus-ttd). Still other issues are surfacing. After a lot of scrambling I'm inching closer but I still have a problem with strings framing, that does not depend on the version of the context used. Here is an easily reproducible version of the problem. Let's start with a very minimal TD:

{
    "@context": "https://www.w3.org/2022/wot/td/v1.1",
    "@type": "Thing",
    "title": "MyThingTitle"
}

Notice: this TD is not valid since it's missing security related info, but I'm not interested in validation now. Using the JSON-LD Playground and the RDF Distiller, the roundtripping works just fine. Using Titanium I get the following end result:

{
    "@type": "Thing",
    "td:title": {
        "@value": "MyThingTitle",
        "@type": "http://www.w3.org/1999/02/22-rdf-syntax-ns#langString"
    },
    "@context": "https://www.w3.org/2022/wot/td/v1.1"
}

By trying to understand what's going on I seen that in the context you can find:

      "title": {
        "@id": "td:title",
        "@language": "en"
      },

Let's see what happens. Both Titanium and JSON-LD playground produce the following RDF triples for the TD:

_:b0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://www.w3.org/2019/wot/td#Thing> .
_:b0 <https://www.w3.org/2019/wot/td#title> "MyThingTitle"@en .

So far so good. Now let's look at the unframed RDF -> JSON. Here RDF Distiller's output (that also perform compaction):

{
  "@id": "_:b0",
  "@type": "https://www.w3.org/2019/wot/td#Thing",
  "https://www.w3.org/2019/wot/td#title": {
    "@language": "en",
    "@value": "MyThingTitle"
  }
}

And that is Titanium (uncompacted):

[
    {
        "@id": "_:b0",
        "@type": [
            "https://www.w3.org/2019/wot/td#Thing"
        ],
        "https://www.w3.org/2019/wot/td#title": [
            {
                "@value": "MyThingTitle",
                "@type": "http://www.w3.org/1999/02/22-rdf-syntax-ns#langString"
            }
        ]
    }
]

Both use a value type for the title, but Titanium loses the language info and adds a @type. It seems like, given the fact that in RDF the object is "MyThingTitle"@en (an RDF plain literal with language tags) and not just a "plain literal", Titanium decided that this is not a "basic" RDF string that can be mapped to a "basic" JSON string but it is a richer data type for which it wants to preserve information that would be lost otherwise (but it fails, given the fact that the only additional information, the language tags, is lost anyway). The Ruby library, on the other side, decided to map the RDF string to a JSON string and maintain the language tag part by adding "@language": "en". Now, the behavior of the Ruby library seems much more reasonable to me, yet I've not been able to find a point in the JSON-LD recommendation that explicitly states how language tags should be converted in these cases. If I'm sure that what Titanium does is wrong, I could open an issue. I think I will try that anyway...

DavideRossi commented 3 weeks ago

Oh, and I forgot a bit (this thing is a bloody mess). You can force Titanium to behave like the Ruby/Javascript libs with strings by setting setUseNativeTypes to false. But then, all your boolean properties become like this:

            "observable": {
                "@value": "false",
                "type": "xsd:boolean"
            }

So you either have messed up booleans or messed up strings!

DavideRossi commented 3 weeks ago

Oh well. These were two bugs in Titanium JSON-LD. One was an issue with RDF strings that are stored as such when they have a language tag and are stored as JSON strings when they don't, and another was a compaction bug, where the type context is not updated in sub-contexts. Took me a couple of days to figure that out. It also means that all Java-based thing directories out there should suffer from the same problem. I wonder how @AndreaCimminoArriaga managed that in WotHive...

wiresio commented 3 hours ago

@DavideRossi - sorry for the late reaction: Is the summary that you have / had challenges with the Titanium stack or do you thing that there is (still) an issue with the TD context?

DavideRossi commented 2 hours ago

Yes, the issues I was experiencing were due to a couple of bugs in Titanium JSON-LD. I have fixed them in the code I use and it works just fine. That being said, I only tested it with rather elementary TDs, I can't rule out that other context related issues exist.