Add JSON-LD Support

[JordanShurmer]

This is a feature request. I'll gladly contribute the work when I get the time, but I figured I should go ahead and open the issue in case others want to contribute it. Or maybe there are existing solutions.

Background

This crate seems to be the most mature and most recently maintained RDF related crates. So, I'm hoping to use this in an LDP server I'm creating. However, JSON-LD is a required format in LDP servers.

The Request

• Add JSON-LD as a supported format for parsing and serializing graphs

[pchampin]

This is indeed something I'd like to add -- especially since I am involved in JSON-LD 1.1 :-)

JSON-LD 1.1 is about to enter the CR period, which means that the spec is now stable, and implementations are called for.

I'm aware of three projects in Rust (by decreasing order of recent activity):

• https://github.com/lo48576/json-ld (looking more active)
• https://github.com/kroeg/jsonld-rs
• https://github.com/ct0r/jsonld.rs

I'm willing to contribute to the first one, in order to help bring it to full compliance. Feel free to do so too, the more the merrier :-)

Once it is functional, I definitely plan to integrate it to sophia. Here again, any help is welcome.

[JordanShurmer]

ok, awesome! glad to see things are all generally moving forward and what not. I'll start contributing as I can :)

[JordanShurmer]

I had not seen the 1.1 spec work before, so I'll be checking that out too.

[pchampin]

FYI, 1bad729 indtroduced the crate sophia_jsonld.

Ir is very incomplete at the moment (only supports the serialization from RDF algorithm), but that's a start...

[alexkreidler]

I may start work on this soon. As it seems Sophia, Rio, and Oxigraph are very focused on performance, I think it may be beneficial to create two submodules of sophia_jsonld: one like full that fully implements the spec, and another streaming one, based on the following spec and implementations: https://www.w3.org/TR/2020/NOTE-json-ld11-streaming-20200507/ https://github.com/rubensworks/jsonld-streaming-serializer.js https://github.com/rubensworks/jsonld-streaming-parser.js

I will probably look at other JSON parsers in rust to get some inspiration. I'm a little inclined to just built the system on top of an existing well-maintained JSON parser like serde's StreamDeserializer. I've also always liked the Parsing Expression Grammar formalism as it makes things simple, but looking at the Rust libraries it seems they are slower than other options. Nom looks like the next-best thing. However I am a bit wary of rolling my own entire parsing system; I don't think I trust my Rust skills that much yet. As long as I stay within a tool that has documentation, I'll be able to figure things out.

I'd appreciate any thoughts or feedback people had before I jump in!

[Tpt]

It looks like a great idea! +1 to reuse a steaming JSON parser. It's what we did for the RDF/XML parsers with quick_xml. Here is a JSON parser benchmark.

I would love having rio_jsonld crates if you want to host your work as part of Rio!

[pchampin]

@alexkreidler agreed, a streaming JSON-LD parser/serializer would be very nice, in addition to a standard one. For the parser part, integrating it with Rio looks like the way to go, as Sophia is relying on Rio for all its parsers now.

[LuisOsta]

@pchampin Based on the current support for Json-ld (https://github.com/pchampin/sophia_rs/tree/main/jsonld) do you know how we could convert a JSON-LD document to a RDF DataSet? I am working on implementing URDNA2015 (https://w3c-ccg.github.io/rdf-dataset-canonicalization/spec/#canonicalization-algorithm) and while I would love to utilize Sophia for the RDF functionality I can't seem to figure out how to handle the conversion from JSON-LD to RDF with Sophia

[pchampin]

yes, implementing JSON-LD to RDF was in my plans. Your comment bumped it up my priority list ;)

[LuisOsta]

Okay awesome, if there's anyway I can help let me know! I'm happy to contribute if needed

[pchampin]

@LuisOsta finally I managed to come up with something. It's in commit 50980bf.

As can be seen in the commit message, it still have a number of limitations. Regarding the first one (requires input in expanded form), a library such as https://github.com/timothee-haudebourg/json-ld can be use to expand arbitrary JSON-LD before passing it to Sophia. The reason why I didn't integrate it into Sophia is that this library requires nightly features, and I prefer Sophia to be available on stable Rust. But hopefully, the feature that json-ld depends on (Generic Assiociated Types) should soon land in stable Rust.

The other limitations require a little more work on the parser it self. Any PR is welcome ;)

[LuisOsta]

Awesome, and I think that the json-ld library will hopefully be stabilized soon -https://github.com/timothee-haudebourg/json-ld/issues/39. Let me review the commit you linked

[LuisOsta]

@pchampin The JSON-LD library has now been stabilized in https://github.com/timothee-haudebourg/json-ld/issues/39 with the latest stable version being 0.9.1 IRRC.

[pchampin]

@LuisOsta yes, that's on my radar, but thanks for the ping

[pchampin]

@LuisOsta I just commited a first implementation of RDF C14N, based on https://www.w3.org/TR/rdf-canon . JSON-LD is the next item on my list...

[LuisOsta]

That's great to hear! Also @pchampin I just wanted to get a better understanding of what JSON-LD support would be like.

Would Sophia support the URDNA2015 RDF normalization algorithm described in https://w3c.github.io/rdf-canon/spec/. (Which we would use in order to create the Data Integrity Proofs needed to create JSON-LD Verifiable Credentials.

Or would Sophia be something we would use to implement the URDNA2015 RDF normalization algorithm ourselves?

Just want to make sure we can prepare accordingly. Would love if it was the former but completely understand if its gonna be the latter.

[pchampin]

just wanted to get a better understanding of what JSON-LD support would be like.

Sophia is an RDF library. It allows you to handle RDF graphs and datasets that conform to the RDF abstract syntax (actually a superset of it).

Sophia provides a number of parsers and serializers for, well, parsing graphs/datasets from different formats, and serializing them back in those formats. It currently supports RDF/XML, N-Triples, N-Quads, Turtle, TriG. JSON-LD support will come as yet another parser and yet another serializer. The serializer will allow options for specifying a specific context, so that you can control how the produced JSON looks. It will be based on timothee-haudebourg/json-ld.

Would Sophia support the URDNA2015 RDF normalization algorithm described in https://w3c.github.io/rdf-canon/spec/. (Which we would use in order to create the Data Integrity Proofs needed to create JSON-LD Verifiable Credentials.

Or would Sophia be something we would use to implement the URDNA2015 RDF normalization algorithm ourselves?

Since yesterday, Sophia provides a function that accepts any Dataset, and produces the normalized nquads as a String. So when we support JSON-LD, you will be able to load any JSON-LD document into a Dataset, and normalize that dataset as you would do with jsonld.js or pyld.

Finally, notice that:

• I'm talking about the tip of the main branch on github, this is not released on crates.io yet
• The preliminary partial support for JSON-LD that I announced [a few months ago] is no longer available on the github version, because I went through a big refactoring, and rather than porting that hack, I'm waiting to find the time to properly integrate timothee-haudebourg/json-ld

Hope this helps

[LuisOsta]

Okay awesome that makes perfect sense, thank you so much!

[pchampin]

Closing, as JSON-LD support is now integrated on 0.8.0-alpha (and soon to be released, hopefully)