named contexts - Githubissues

Description Our api needs to accept json-ld in order to properly function. Any operation can come wrapped in a verifiable credential, and in order for the signature on the credential to be useful the credential subject needs to be proper json-ld that can be canonized and hashed.

Problem statement In order for json-ld to be easy to use, users need to be able to use @context to shorten their iris. We currently allow a default context but it's use has a few drawbacks. Users don't have to specify an @context on every request, but:

default context is implicit and not compatible with verifiable credentials, since the context needs to be properly embedded within the credential subject and cannot be imposed externally without invalidating the signature.
default context is not stored in a way that is documented for users, so viewing or updating default contexts is unwieldy.
operations are not stored with their context, so expanding requires finding the default context. This can make troubleshooting more difficult.

Approach We can store contexts in the ledger by associating them with an iri:

{"@id" <context-name-iri>
 "http://flur.ee/ns/context" {"@type" "@json" "@value" <context goes here>}}

Now a user can query for a context like any other data.

Note: our json-ld processor does not currently support @type @json, so a naive implementation would have to store a json string.

In order to allow a user to reference a context from an operation, like this:

{"@context" <context-name-iri> ...}

We would need to supply a custom document loader to our json-ld processor. Fortunately, the official api has specified how such a pluggable loader should work, and it is straightforward to implement.

The last step would be to update our call sites to use the custom document loader:

(json-ld/expand <json-ld> {:document-loader (fluree-loader db)})

where fluree-loader is defined as

(defn fluree-loader
  [db]
  (fn [url options]
    @(fluree/query db {"select" "?context"
                       "where" [[url "http://flur.ee/ns/context" "?context] ]})))

This approach offers some benefits over default context:

adding context is explicit, users have to store the context themselves. This makes it easier for them to search and update.
we now have a formal method for immutable contexts, something that has been a challenge for json-ld in the wild.

With a drawback:

users have to include the fluree context on every operation in order for them to be valid json-ld, at least if they want to wrap them in a credential

@(fluree/query db {"@context" "http://flur.ee/"
                               "select" {"?s" ["*"]}
                                "where" [["?s" "ex:name" "Dan"]]})

Follow-on

we could, as a convenience, supply the actual context for common context urls at ledger creation.

@(fluree/create "myledger" {"contexts" ["http://schema.org/" "http://www.w3.org/ns/shacl"]})

Would include the following data in the bootstrap:

[{"@id" "http://schema.org/"
  "f:context" <schema.org context>}
{"@id" "http://www.w3.org/ns/shacl"
  "f:context" <shacl context>}]

Or we could just include all of our recommended default contexts as part of the fluree context, there are multiple ways we could approach the convenience of default context.

This approach could also live in parallel with default context, though I think having both approaches could be confusing from a user standpoint.

Here's a quick look at what the usage could look like:

(comment
  (def fluree-context
    {"f" "http://flur.ee/ns/"
     "commitDetails" {"@id" "f:commitDetails" "@type" "@json"}
     "delete" {"@id" "f:delete" "@type" "@json"}
     "depth" {"@id" "f:depth" "@type" "@json"}
     "filter" {"@id" "f:filter" "@type" "@json"}
     "groupBy" {"@id" "f:groupBy" "@type" "@json"}
     "having" {"@id" "f:having" "@type" "@json"}
     "history" {"@id" "f:history" "@type" "@json"}
     "insert" {"@id" "f:insert" "@type" "@json"}
     "context" {"@id" "f:context" "@type" "@json"}
     "limit" {"@id" "f:limit" "@type" "@json"}
     "offset" {"@id" "f:offset" "@type" "@json"}
     "opts" {"@id" "f:opts" "@type" "@json"}
     "orderBy" {"@id" "f:orderBy" "@type" "@json"}
     "select" {"@id" "f:select" "@type" "@json"}
     "selectDistinct" {"@id" "f:selectDistinct" "@type" "@json"}
     "selectOne" {"@id" "f:selectOne" "@type" "@json"}
     "t" {"@id" "f:t" "@type" "@json"}
     "values" {"@id" "f:values" "@type" "@json"}
     "where" {"@id" "f:where" "@type" "@json"}})

  (def conn @(fluree/connect {:method :memory}))
  (def ledger @(fluree/create conn "credentialtest"))
  (def db0 (fluree/db ledger))

;; have to use `json/stringify` because we don't currently support `@type` `@json`
  (def db1 @(fluree/stage db0 [{"@id" "ledger:credentialtest/context"
                                "http://flur.ee/ns/context"
                                {"@type" "@json"
                                 "@value" (json/stringify {"@context"
                                                           {"@base" "ledger:credentialtest/"
                                                            "@vocab" "ns/"}})}}
                               {"@id" "http://flur.ee/ns/fluree-context"
                                "http://flur.ee/ns/context"
                                {"@type" "@json"
                                 "@value" (json/stringify {"@context" fluree-context})}}]))

  (defn fluree-loader
    [db]
    (fn [url options]
      @(fluree/query db {"selectOne" "?context"
                         "where" [[url "http://flur.ee/ns/context" "?context"]]})))

  (jld-processor/expand {"@context" ["http://flur.ee/ns/fluree-context" {"schema" "http://schema.org/"
                                                                         "ex" "http://example.com/ns/"}]
                         "@graph"
                         [ ;; query
                          {"select-distinct" ["?name" "?email" {"?s" ["*"]}]
                           "where" [["?s" "schema:name" "?name"]
                                    ["?s" "schema:email" "?email"]
                                    ["?s" "ex:favNums" "?favNum"]]
                           "order-by" "?favNum"
                           "values" ["?name" ["Marcela" "Ben" "Wes"]]}
                          ;; update
                          {"where" [["?s" "ex:x" "foo-cat"]]
                           "delete" ["?s" "ex:x" "foo-cat"]
                           "insert" ["?s" "ex:x" "foo-dog"]}
                          ;; history
                          {"history" [nil "ex:x" "foo-cat"] "t" {"from" 1}}]}
                        {:document-loader (fluree-loader db1)})
;; =>
  [{"http://flur.ee/ns/where"
    [{"@value"
      [["?s" "schema:name" "?name"]
       ["?s" "schema:email" "?email"]
       ["?s" "ex:favNums" "?favNum"]],
      "@type" "@json"}],
    "http://flur.ee/ns/values"
    [{"@value" ["?name" ["Marcela" "Ben" "Wes"]], "@type" "@json"}]}
   {"http://flur.ee/ns/where" [{"@value" [["?s" "ex:x" "foo-cat"]], "@type" "@json"}],
    "http://flur.ee/ns/delete" [{"@value" ["?s" "ex:x" "foo-cat"], "@type" "@json"}],
    "http://flur.ee/ns/insert" [{"@value" ["?s" "ex:x" "foo-dog"], "@type" "@json"}]}
   {"http://flur.ee/ns/history" [{"@value" [nil "ex:x" "foo-cat"], "@type" "@json"}],
    "http://flur.ee/ns/t" [{"@value" {"from" 1}, "@type" "@json"}]}]

  ,)

Our initial version stored the default context as data, we deliberately removed it and instead treated it as commit metadata for reasons that were documented in the tickets where that work happened.

I thought we discussed defining queries as type @json, which we need to add to the parser anyhow and should be trivial. Is there a reason to prefer this over that approach? That should fully address the query use case without any of the complexities described above.

As for signed transactions, I think we give two options, (a) a new property of type @json which you can put the entire transaction in, or (b) if you want the transaction to be full JSON-LD then the HTTP-API should expose a URL for every ledger's context, then that URL can be used by any JSON-LD library and dynamically pull in the context. Ultimately the context that gets used will be the one in the system, so if that gets updated while the credential is "in-flight", it would ultimately reject the signature - so it is safe. This would not require any custom loaders.

fluree / core

named contexts #16