decentralized-identity / confidential-storage

Confidential Storage Specification and Implementation
https://identity.foundation/confidential-storage/
Apache License 2.0
78 stars 23 forks source link

Delete This Issue Once this Appendix has been merged into the identity hubs section #128

Open OR13 opened 3 years ago

OR13 commented 3 years ago
<section class="informative appendix">
      <h2>
Identity Hubs
      </h2>
      <p>
Hubs let you securely store and share data. A Hub is a datastore containing semantic data objects at well-known locations.  Each object in a Hub is signed by an identity and accessible via a globally recognized API format that explicitly maps to semantic data objects.  Hubs are addressable via unique identifiers maintained in a global namespace.
      </p>
      <section>
        <h3>
One DID to Many Hub Instances
        </h3>
        <p>
A single entity may have one or more instances of a Hub, all of which are addressable via a URI routing mechanism linked to the entity's identifier.  Hub instances sync state changes, ensuring the owner can access data and attestations from anywhere, even when offline.
        </p>
        <section>
          <h4>
DID Document Service Endpoint Descriptors
          </h4>
          <p>
There are two different variations of Hub-specific DID Document Service Endpoint descriptors, one that users associate with their DIDs, and another that Hosts use to direct requests to locations where their Hub infrastructure resides.
          </p>
          <p>
Users specify their Hub instances (different Hub Hosts) via the <code>UserServiceEndpoint</code> descriptor:
          </p>
          <pre class="example">
  "service": [{
  "type": "IdentityHub",
  "publicKey": "did:foo:123#key-1",
  "serviceEndpoint": {
    "@context": "schema.identity.foundation/hub",
    "@type": "UserServiceEndpoint",
    "instances": ["did:bar:456", "did:zaz:789"]
  }
}]
          </pre>
          <p>
Hosts specify the locations of their Hub offerings via the <code>HostServiceEndpoint</code> descriptor:
          </p>
          <pre class="example">
"service": [{
  "type": "IdentityHub",
  "publicKey": "did:bar:456#key-1",
  "serviceEndpoint": {
    "@context": "schema.identity.foundation/hub",
    "@type": "HostServiceEndpoint",
    "locations": ["https://hub1.bar.com/.identity", "https://hub2.bar.com/blah/.identity"]
  }
}]
          </pre>
        </section>
        <section>
          <h4>
Syncing data between Hubs
          </h4>
          <p>
Hub instances must sync data without requiring master-slave relationships or forcing a single implementation for storage or application logic. This requires a shared replication protocol for broadcasting and resolving changes. The protocol for reproducing Hub state across multiple instances is in the formative phases of definition/selection, but should be relatively straightforward to integrate on top of any NoSQL datastore.
          </p>
        </section>
        <section>
          <h4>
Hub data serialization and export
          </h4>
          <p>
All Hubs must support the export of their serialized state. This is to ensure the user retains full control over the portability of their data. A later revision to this document will specify the process for invoking this intent and retrieving the serialized data from a Hub instance.
          </p>
        </section>
      </section>
      <section>
        <h3>
Hub Protocol Schemes
        </h3>
        <section>
          <h4>
Hub URI Scheme
          </h4>
          <p>
In addition to the URL path convention for individual Hubs instances, it is important that links to an identity owner's data not be encoded with a dependency on a specific Hub instance. To make this possible, we propose the introduction of the following Hub URI scheme:
          </p>
          <pre class="example">
hub://did:foo:123abc/
          </pre>
          <p>
User Agents that understand this scheme will leverage the Universal Resolver to lookup the Hub instances of the target DID and address the Hub endpoints via the Service Endpoints it finds within. This allows the formation of URIs that are not Hub instance-specific, allowing a more natural way to link to a DID's data, without having to specify a specific instance. This also prevents the circulation of dead links across the Web, given an identity owner can choose to add/remove new Hub instances at any time.
          </p>
        </section>
      </section>
      <section>
        <h3>
Authentication
        </h3>
        <p>
The process of authenticating requests to identity hubs will follow the DIF/W3C DID Auth schemes. These standards are in early phases of formation - <a href="https://github.com/WebOfTrustInfo/rebooting-the-web-of-trust-spring2018/blob/master/draft-documents/did_auth_draft.md">more info is available here</a>.
        </p>

        <p>
The current Identity Hub authentication scheme seeks to accomplish two tasks:
        </p>
        <ul>
          <li>
mutually authenticate the client and the Hub using each's respective DID and associated keys.
          </li>
          <li>
encrypt all Hub requests and responses such that their contents are only available to the holders of the DID keys involved in the message exchange.
          </li>
        </ul>
        <p>
The current authentication scheme is an implementation of DID Auth, <a
href="https://gitHub.com/WebOfTrustInfo/rwot6-santabarbara/blob/master/final-documents/did-auth.md">as
described here</a>. This document will outline how to authenticate Hub requests
and responses. For full details on the authentication protocol used, and for a
reference implementation of the protocol, please refer to the <a
href="https://gitHub.com/decentralized-identity/did-auth-jose/blob/master/docs/Authentication.md">`did-auth-jose`
library</a>.
        </p>

        <section>
          <h4>
Authenticating a Hub request
          </h4>
          <p>
Identity Hub requests and responses are signed and encrypted using the DID keys
of the sender and the recipient. This protects the message over any
transportation medium. All encrypted requests and responses follow the
<a href="https://tools.ietf.org/html/rfc7516">JSON Web Encryption (JWE)
standard</a>.
          </p>
          <p>
The steps to construct a JWE are as follows. First, construct a JWT. This JWT
will be signed by the sender of the Hub request; the `iss`. This JWT must have
the following form:
          </p>

          <pre class="example">
// JWT headers
{
  "alg": "RS256",
  "kid": "did:example:abc123#key-abc",
  "did-requester-nonce": "randomized-string",
  "did-access-token": "eyJhbGciOiJSUzI1N...",
}

// JWT body
{
  "@context": "https://schema.identity.foundation/0.1",
  "@type": "WriteRequest",
  "iss": "did:example:abc123",
  ...
}

// JWT signature
uQRqsaky-SeP3m9QPZmTGtRtMoKzyg6wwWF...
          </pre>
          <p>
The JWT body is just the request whose format is outlined in
<a href="#identity-hubs"></a>. The header values must be the following:
          </p>

          <table class="simple">
            <tr>
              <th>
Header
              </th>
              <th>
Description
              </th>
            </tr>
            <tr>
              <td>
`alg`
              </td>
              <td>
Standard JWT header. Indicates the algorithm used to sign the JWT.
              </td>
            </tr>
            <tr>
              <td>
`kid`
              </td>
              <td>
Standard JWT header. The value should take the form `{did}#{key-id}`. Another
app can take this value, resolve the DID, and find the indicated public key that
can be used for signature validation of the commit. Here we have used
`did:example:abc123`, because the request is signed with the user's private key.
              </td>
            </tr>
            <tr>
              <td>
`did-requester-nonce`
              </td>
              <td>
A randomly generated string that must be cached on the client side. This string
will be used to verify the response from the Hub in the sections below.
              </td>
            </tr>
            <tr>
              <td>
`did-access-token`
              </td>
              <td>
A token that should be cached on the client side and included in each request
sent to the Hub. Since we do not have an access token yet, leave this property
out on the initial request. Sections below explain how to get an access token.
              </td>
            </tr>
          </table>
          <p>
This JWT must use the typical JWT compact serialization format.
          </p>
          <p>
We can now use this JWT as the plaintext used to construct our JWE. The JWE must
have the following format.
          </p>

          <pre class="example">
// JWE protected header
{
  "alg": "RSA-OAEP-256",
  "kid": "did:example:abc456#abc-123",
  "enc": "A128GCM",
}

// JWE encrypted content encryption key
OKOawDo13gRp2ojaHV7LFpZcgV7T6DVZKTyKOM...

// JWE initialization vector
48V1_ALb6US04U3b...

// JWE plaintext (the JWT from above)
eyJhbGciOiJSUzI1NiIsImtpZCI6InRlc3R...

// JWE authentication tag
XFBoMYUZodetZdv...
          </pre>

          <p>
We strongly reccommend using a JWT library to produce the above JWE. Using a
library, you should only need to provide the protected headers and the
plaintext. The plaintext value should be the JWT constructed above. The header
values are:
          </p>

          <table class="simple">
            <tr>
              <th>
Header
              </th>
              <th>
Description
              </th>
            </tr>
            <tr>
              <td>
`alg`
              </td>
              <td>
Standard JWT header. Indicates the algorithm used to encrypt the JWE content
encryption key.
              </td>
            </tr>
            <tr>
              <td>
`kid`
              </td>
              <td>
Standard JWT header. The value should take the form `{did}#{key-id}`. Indicates
the Hub's key that is used to encrypt the JWE content encryption key. Here we
have used `did:example:abc456`, since that is the DID used by the Hub. The DID
used here should match the `aud` value in the Hub `WriteRequest`.
              </td>
            </tr>
            <tr>
              <td>
`enc`
              </td>
              <td>
Standard JWT header. Indicates the algorithm used to encrypt the plaintext using
the content encryption key to produce the ciphertext and authentication tag.
              </td>
            </tr>
          </table>
          <p>
Finally, you have a signed and encrypted Hub request that can be transmitted to
the user's Identity Hub for secure storage.
          </p>
        </section>
        <section>
          <h4>
Caching the access token
          </h4>
          <p>
To send a successful request to an Identity Hub, you need to include an access
token in the `did-access-token` header of the JWE. The access token is a
short-lived JWT that can be used across many Hub requests until it expires.
          </p>
          <p>
On an initial request to an Identity Hub, you should exclude the
`did-access-token` header. When a Hub request does not include this header, the
Hub will reject the request. Instead, the Hub will return a JWE response (as
described in the next section) whose payload is an access token. You should
extract the access token from the response and cache it somewhere safe. The
access token can be used for subsequent requests.
          </p>
          <p>
Once you've cached the access token, include it in each request in the
`did-access-token` JWE header as described above.
          </p>
          <p>
Eventually, the access token will expire. Its expiry time can be found in the
`exp` claim inside the access token. If you attempt to use an expired access
token, the Identity Hub will return an error indicating a new access token is
required. To get a new access token, send another hub request without the
`did-access-token` header.
          </p>
        </section>
        <section>
          <h4>
Receiving a hub response
          </h4>
          <p>
When possible, a hub will respond with a JWE encrypted with the client's DID
keys:
          </p>
          <pre class="example">
eyJhbGciOiJSU0EtT0FFUCIsImVuYyI6IkEyNTZHQ00ifQ...
          </pre>
          <p>
This JWE can be decrypted with the client's private key following the JWE
standard to reproduce the response's plaintext.
          </p>
          <p>
The contents of the JWE will either be a valid hub response or a new access
token. A new access token will only be included if the `did-access-token` header
was omitted in the request.
          </p>
        </section>
      </section>
      <section>
        <h3>
Authorization
        </h3>
        <p>
Access control for data stored in Hubs is currently implemented via a bare bones
permission layer. Access to data can be granted by a Hub owner, and can be
restricted to certain types of data. More features to improve control over data
access will be added in the future.
        </p>
        <p>
The success of a decentralized identity platform is dependent upon the ability
for users to share their data with other people, organizations, apps, and
services in a way that respects and protects a user's privacy. In our
decentralized platform, all user information & data resides in the user's
identity Hub. This section outlines the current proposal for identity hub
authorization.
        </p>

        <section>
          <h4>
Scope of the current design
          </h4>
          <p>
This proposal is a first cut. The intention is to start extremely simple, and
extend the model to include more richness over time. We choose to focus on two
simple use cases, described below.
          </p>

          <section>
            <h5>
Use case 1: Registering for a website
            </h5>

            <blockquote>
Alice has added some useful data about her wardrobe style to her Hub: her
measurements from her tailor, and a list of her favorite clothing brands. When
Alice goes to try out a new online clothing retailer, the retailer's website
allows her to set up an account using her DID. After signing in her DID, the
retailer's website is able to access Alice's style data. Alice does not have to
re-enter her sizes in the site, and the site can give her recommended options
based on her brand preferences.
            </blockquote>

            <figure>
              <img src="diagrams/permissions-use-case-1.png" />
              <figcaption>
Permission request flow
              </figcaption>
            </figure>
          </section>

          <section>
            <h5>
Use case 2: Reviewing & managing access
            </h5>
            <blockquote>
Alice learns that one of the websites she visited is making improper use of her
personal data. She would like to immediately remove that website's access to her
Hub.
            </blockquote>

            <figure>
              <img src="diagrams/permissions-use-case-2.png" />
              <figcaption>
Permission denied flow
              </figcaption>
            </figure>
          </section>

          <section>
            <h5>
Out of scope
            </h5>
            <p>
These use cases, and the current Hub authorization system are not sufficient to
consider identity Hubs ready for real world usage. It leaves out several
features that have been discussed as being necessary for a minimally viable
authorization layer, including:
            </p>
            <p>
<strong>Features that control <em>what</em> is being granted:</strong>
            </p>
            <ul>
              <li>
How to grant a permission to a specific object by ID, rather than all objects of
a certain type.
              </li>
              <li>
How to grant a permission to a property of some object type, rather than the
entire object.
              </li>
              <li>
How to grant permission to an object type and all of the children object types
in its respective schema.
              </li>
              <li>
How to filter a permission to only:
                <ul>
                  <li>
objects created by a specific DID.
                  </li>
                  <li>
objects created in a certain time period.
                  </li>
                  <li>
objects larger than some byte size.
                  </li>
                </ul>
              </li>
              <li>
How to grant a permission to a zero-knowledge proof of some object, rather than the entire object.
              </li>
              <li>
How to grant permission to act as a delegate of a DID when interacting with other Hubs.
              </li>
            </ul>

            <p>
<strong>Features that control <em>who</em> is being granted access:</strong>
            </p>
            <ul>
              <li>
How to grant a permission to all DIDs, and therefore make some data public.
              </li>
              <li>
How to create a permission that explicitly denies a DID access to an object.
              </li>
            </ul>

            <p>
<strong>Features that limit/expand <em>where or when</em> access is granted:</strong>
            </p>
            <ul>
              <li>
How to time-bound permissions, such that a permission expires automatically.
              </li>
              <li>
How to grant permissions to an app on some devices, but not others.
              </li>
            </ul>

            <p>
<strong>Features that control <em>why</em> access is granted:</strong>
            </p>
            <ul>
              <li>
How an app can specify why permission is being requested.
              </li>
              <li>
How a user can specify why permission is being denied.
              </li>
              <li>
How relying parties and trust providers are reviewed for trustworthiness and integrity.
              </li>
            </ul>

            <p>
<strong>Features that are related to Hub authorization, but will be addressed at a later time:</strong>
            </p>
            <ul>
              <li>
How to request & send callbacks to notify apps of changes to data and
permissions in a Hub.
              </li>
              <li>
How to authorize the execution of services, or extensions, in a Hub.
              </li>
              <li>
What format(s) the Hub uses for requests & responses.
              </li>
              <li>
How to encrypt data in a Hub such that the Hub provider cannot access it.
              </li>
            </ul>

            <p>
Clearly, there is a large body of functionality that can be added to Hub
authorization over time. This is why this initial document intentionally strives
to be as simple as possible. We'll incorporate these things into Hub
authorization over time as we receive feedback from early adopters of Identity
Hubs.
            </p>
          </section>
        </section>

        <section>
          <h4>
Authorization Model
          </h4>
          <p>
Access to data in Identity Hubs is controlled by a special object stored in Hubs
called a `PermissionGrant`. The structure of a `PermissionGrant` is:
          </p>

          <pre class="example">
{
  "owner": "did:example:12345", // the identity owner (granters)'s DID
  "grantee": "did:example:67890", // the grantee's DID
  "context": "schemas.clothing.org", // the data schema context
  "type": "measurements", // the data type
  "allow": "-R--", // allows create, read, update, and delete
  ... // additional richness & specificity can be added in the future
}
          </pre>

          <section>
            <h5>
Granting permissions
            </h5>
            <p>
When a hub owner grants a permission to another DID, they can do so by
specifying the exact objects in the permission grant. When permissions span more
than one data type, several PermissionGrant objects can be created. For each
PermissionGrant, the following object should be written to the `Permissions`
interface of the owner's Hub, typically via user agent:
            </p>
            <pre class="example">
{
  "@context": "schema.identity.foundation/Hub/",
  "@type": "PermissionGrant",
  "owner": "did:example:12345",
  "grantee": "did:example:67890",
  "context": "schemas.clothing.org",
  "type": "measurements",
  "allow": "-R--"
}
            </pre>
            <p>
Note that the Hub Permissions interface only supports the single PermissionGrant
object type. The Hub should reject any requests to create objects of other types
in the Permissions interface, barring future updates to the PermissionGrant
model.
            </p>
            <p>
The response format, and any error conditions, should be consistent with all
other requests to Hubs. Upon creation of this permission grant object in a
user's Hub, the permission will be propagated to all other Hub instances listed
in the user's DID document via the Hub's standard sync & replication protocol.
This will ensure that all Hub instances are up-to-date with all new permission
grants in a timely manner.
            </p>
          </section>
          <section>
            <h5>
Checking permissions
            </h5>
            <p>
The following describes the logic implemented by the Hub's authorization layer
when a request arrives.
            </p>

            <figure>
              <img src="diagrams/permissions-verification.png"/>
              <figcaption>
              </figcaption>
            </figure>

            <ol>
              <li>
Receive incoming request from client
              </li>
              <li>
Determine relevant schema, verb, and client from request
              </li>
              <li>
Query for all PermissionGrants that whose object_type matches the schema, for
the given client DID
              </li>
              <li>
Check if any query results allow the verb in question
              </li>
              <li>
Return success/failed authorization check
              </li>
            </ol>
            <p>
Note that PermissionGrants do not understand or evaluate the structure of a
given schema. For instance, if a user grants access to all
"https://schema.org/game" objects, they <strong>do not</strong> implicitly grant
access to all "https://schema.org/videogame" objects (which is a child of game
in schema.org's hierarchy).
            </p>
          </section>
          <section>
            <h5>
Reviewing & managing permissions
            </h5>

            <p>
`PermissionGrant` objects can be created, read, modified, and deleted just like
any other object in a hub. To revoke access to data, the Hub owner needs to
simply modify an existing `PermissionGrant` or delete it entirely. Instructions
for reading and writing data in Identity Hubs is available in
<a href="#api"></a>.
            </p>
          </section>
          <section>
            <h5>
Requesting permissions
            </h5>
            <p>
At this time, proposals for how to request access to data in an identity hub via
a user agent are still being evaluated. In the future, we will update this
document with details on how a client can request access from a user.
            </p>
          </section>
        </section>

      </section>
      <section>
        <h3>
API
        </h3>
        <p>
Because of the sensitive nature of the data being transmitted to Identity Hubs,
the Identity Hub request and response API may look a bit different to developers
who are used to a traditional REST service API. Most of the differences are
based on the high level of security and privacy decentralized identity
inherently demands.
        </p>
        <section>
          <h4>
Commits
          </h4>
          <p>
All data in identity hubs is represented as a series of "commits". A commit is
similar to a git commit; it represents a change to an object. To write data to
an identity hub, you need to construct and send a new commit to the hub. To read
data from an identity hub, you need to fetch all commits from the hub. An
object's current value can be constructed by applying all its commits in order.
          </p>
          <p>
The use of commits to represent data in identity hubs offers a few distinct
advantages:
          </p>
          <ul>
            <li>
it facilitates the hub's replication protocol, enabling multiple hub instances
to sync data.
            </li>
            <li>
it creates an auditable history of changes to an object, especially when each
commit is signed by a DID.
            </li>
            <li>
it eases implementation for use cases that need offline data modification and
require conflict resolution.
            </li>
          </ul>
          <p>
Each commit in a hub is a
<a href="https://en.wikipedia.org/wiki/JSON_Web_Token">JWT</a> whose body
contains the data to be written to the hub. Here's an example of a deserialized
and decoded JWT:
          </p>

          <pre class="example">
// JWT headers
{
  "alg": "RS256",
  "kid": "did:foo:123abc#key-abc",
  "interface": "Collections",
  "context": "https://schema.org",
  "type": "MusicPlaylist",
  "operation": "create",
  "committed_at": "2018-10-24T18:39:10.10:00Z",
  "commit_strategy": "basic",
  "sub": "did:bar:456def",

// Example metadata about the object that is intended to be "public"
  "meta": {
    "tags": ["classic rock", "rock", "rock n roll"],
    "cache-intent": "full"
  }
}

// JWT body
{
  "@context": "http://schema.org/",
  "@type": "MusicPlaylist",
  "description": "The best rock of the 60s, 70s, and 80s",
  "tracks": ["..."],
}

// JWT signature
uQRqsaky-SeP3m9QPZmTGtRtMoKzyg6wwWF...
          </pre>
          <p>
The commit is signed by the committer writing the data, in this case
<code>did:foo:123abc</code>. To write the commit into a hub, the committer must
send a Hub write request.
          </p>
        </section>
        <section>
          <h4>
Write Request &amp; Response Format
          </h4>
          <p>
Instead of a REST-based scheme where data like the username, object types, and
query strings are present in the URL, Identity Hubs requests are self-contained
message objects that encapsulate all they need to be shielded from observing
entities during transport.
          </p>
          <p>
Each Hub request is a JSON object which is then signed and encrypted as outlined
in the authentication section. The outer envelope is signed with the key of the
"iss" DID, and encrypted with the Hub's DID key(s).
          </p>

          <pre class="example">
{
  iss: 'did:foo:123abc',
  sub: 'did:bar:456def',
  aud: 'did:baz:789ghi',
  "@context": "https://schema.identity.foundation/0.1",
  '@type': 'WriteRequest',

  // The commit in JSON Serialization Format
  // See: https://tools.ietf.org/html/rfc7515#section-3.1
  commit: {
    protected: "ewogICJpbnRlcmZhY2...",

// Optional metadata information not protected by the JWT signature
header: {
  "iss": "did:foo:123abc"
},

payload: "ewogICJAY29udGV4dCI6...",
signature: "b7V2UpDPytr-kMnM_YjiQ3E0J2..."
  }
}
          </pre>
          <p>
Each response is also a JSON object, signed and encrypted in the same way as the request. Its contents are:
          </p>
          <pre class="example">
{
  "@context": "https://schema.identity.foundation/0.1",
  "@type": "WriteResponse",
  "developer_message": "completely optional message from the hub",
  "revisions": ["aHashOfTheCommitSubmitted"]
}
          </pre>
        </section>
        <section>
          <h4>
Object Read Request &amp; Response Format
          </h4>
          <p>
Objects follow one logical object across multiple commits. Object reads do not
contain the literal object data, only metadata associated. Objects may be
queried for using the following request format:
          </p>
          <pre class="example">
{
  "@context": "https://schema.identity.foundation/0.1",
  "@type": "ObjectQueryRequest",
  "iss": "did:foo:123abc",
  "sub": "did:bar:456def",
  "aud": "did:baz:789ghi",
  "query": {
      "interface": "Collections",
      "context": "http://schema.org",
      "type": "MusicPlaylist",

  // Optional object_id filters
  "object_id": ["3a9de008f526d239..", "a8f3e7..."]
  }
}
          </pre>
          <p>
The response to a query for objects returns a list of object IDs along with the
object metadata. The format is:
          </p>
          <pre class="example">
{
  "@context": "https://schema.identity.foundation/0.1",
  "@type": "ObjectQueryResponse",
  "developer_message": "completely optional",
  "objects": [
    {
      // object metadata
      "interface": "Collections",
      "context": "http://schema.org",
      "type": "MusicPlaylist",
      "id": "3a9de008f526d239...",
      "created_by": "did:foo:123abc",
      "created_at": "2018-10-24T18:39:10.10:00Z",
      "sub": "did:foo:123abc",
      "commit_strategy": "basic",
      "meta": {
        "tags": ["classic rock", "rock", "rock n roll"],
        "cache-intent": "full"
      }
    },
    // ...more objects
  ]

// potential pagination token
  "skip_token": "ajfl43241nnn1p;u9390",
}
          </pre>
        </section>
        <section>
          <h4>
Commit Read Request &amp; Response Format
          </h4>
          <p>
To get the actual data in an object, you must read the commits from the Identity
Hub:
          </p>
          <pre class="example">
{
  "@context": "https://schema.identity.foundation/0.1",
  "@type": "CommitQueryRequest",
  "iss": "did:foo:123abc",
  "sub": "did:bar:456def",
  "aud": "did:baz:789ghi",
  "query": {
    "object_id": ["3a9de008f526d239..."],
    "revision": ["abc", "def", ...]
  },
}
          </pre>
          <p>
A response to a query for commits contains a list of commit JWTs:
          </p>
          <pre class="example">
{
  "@context": "https://schema.identity.foundation/0.1",
  "@type": "CommitQueryResponse",
  "developer_message": "completely optional",
  "commits": [
    {
      protected: "ewogICJpbnRlcmZhY2UiO...",
      header: {
        "iss": "did:foo:123abc",
        // Hubs may add additional information to the unprotected headers for convenience
        "rev": "aHashOfTheCommit",
      },
      payload: "ewogICJAY29udGV4dCI6ICdo...",
      signature: "b7V2UpDPytr-kMnM_YjiQ3E0J2..."
    },
    // ...
  ],

// potential pagination token
  "skip_token": "ajfl43241nnn1p;u9390",
}
          </pre>
        </section>

        <section>
          <h3>
Paging
          </h3>
          <p>
<code>skip_token</code> is an opaque token to be used for continuation of a
request.
          </p>
          <p>
They may be returned on responses with multiple results, and added to the
initial request's <code>query</code> object:
          </p>
          <pre class="example">
{
  "iss": "did:foo:123abc",
  "sub": "did:bar:456def",
  "aud": "did:baz:789ghi",
  "@context": "https://schema.identity.foundation/0.1",
  "@type": "ObjectQueryRequest",
  "interface": "Collections",
  "query": {
    "context": "schema.org",
    "type": "MusicPlaylist",
    "skip_token": "ajfl43241nnn1p;u9390"
  }
}
          </pre>
        </section>
      </section>
      <section>
        <h3>
Interfaces
        </h3>
        <p>
To facilitate common interactions and data storage, Hubs provide a set of standard interfaces that can be written to:
        </p>
        <ul>
          <li>
<code>Profile</code> ➜ The owning entity's primary descriptor object (schema
agnostic)
          </li>
          <li>
<code>Permissions</code> ➜ The access control JSON document
          </li>
          <li>
<code>Actions</code> ➜ A known endpoint for the relay of actions to the identity
owner
          </li>
          <li>
<code>Stores</code> ➜ Scoped 1:1 storage space that is directly assigned to
another, external DID
          </li>
          <li>
<code>Collections</code> ➜ The owning entity's identity collections (access
limited)
          </li>
          <li>
<code>Services</code> ➜ any custom, service-based functionality the identity
exposes
          </li>
        </ul></p>

        <section>
          <h4>
Profile
          </h4>
          <p>
Each Hub has a <code>profile</code> object that describes the owning entity. The
profile object should use whatever schema and object that best represents the
entity. To ge the profile for a DID, send an object query to the
<code>Profile</code> interface:
          </p>
          <pre class="example">
{
  "@context": "https://schema.identity.foundation/0.1",
  "@type": "ObjectQueryRequest",
  "iss": "did:foo:123abc",
  "sub": "did:bar:456def",
  "aud": "did:baz:789ghi",
  "query": {
      "interface": "Profile",
  }
}
          </pre>
        </section>
        <section>
          <h4>
Permissions
          </h4>
          <p>
All access and manipulation of identity data is subject to the permissions
established by the owning entity. See <a href="#authorization"></a> explainer
for details.
          </p>
        </section>
        <section>
          <h4>
Actions
          </h4>
          <p>
The <code>Actions</code> interface is for sending a target identity semantically
meaningful objects that convey an intent to the sender, which often involves the
data payload of the object. The <code>Actions</code> interface is not
constrained to simple human-centric communications. Rather, it is intended as a
universal conduit through which identities can transact all manner of
activities, exchanges, and notifications.
          </p>
          <p>
The base data format for conveying an action shall be:
<a href="http://schema.org/Action">http://schema.org/Action</a>
          </p>
          <p>
Here is a list of examples to show the range of use-cases this interface is
intended to support:
          </p>
          <ul>
            <li>
Human user contacts another with a textual message
(<a href="http://schema.org/ReadAction">ReadAction</a>)
            </li>
            <li>
Event app sends a request to RSVP for an event
(<a href="http://schema.org/RsvpAction">RsvpAction</a>)
            </li>
            <li>
Voting agency prompts a user to submit a vote
(<a href="http://schema.org/VoteAction">VoteAction</a>)
            </li>
          </ul>
          <pre class="example">
{
  "@context": "http://schema.org/",
  "@type": "ReadAction",
  "name": "Acme Bank - March 2018 Statement",
  "description": "Your Acme Bank statement for account #1734765",
  "object": PDF_SOURCE
}
          </pre>
        </section>
        <section>
          <h4>
Stores
          </h4>
          <p>
The best way to describe Stores is as a 1:1 DID-scoped variant of the W3C DOM's
origin-scoped <code>window.localStorage</code> API. The key difference being
that this form of persistent, pairwise object storage transcends providers,
platforms, and devices. For each storage relationship between the DID owner and
external DIDs, the Hub shall create a key-value document-based storage area. The
DID owner or external DID can store unstructured JSON data to the document, in
relation to the keys they specify. The Hub implementer may choose to limit the
available space of the storage document, with the option to expand the storage
limit based on criteria the implementer defines.
          </p>
        </section>
        <section>
          <h4>
Collections
          </h4>
          <p>
Data discovery has been a problem since the inception of the Web. Most previous
attempts to solve this begin with the premise that discovery is about individual
entities providing a mapping of their own service-specific API and data schemas.
While you can certainly create a common format for expressing different APIs and
data schemas, you are left with the same basic issue: a sea of services that
can't efficiently interoperate without specific review, effort, and integration.
Hubs avoid this issue entirely by recognizing that the problem with <em>data
discovery</em> is that it relies on <em>discovery</em>. Instead, Hubs assert the
position that locating and retrieving data should be an <em>implicitly
knowable</em> process.
          </p>
          <p>
Collections provide an interface for accessing data objects across all Hubs,
regardless of their implementation. This interface exerts almost no opinion on
what data schemas entities use. To do this, the Hub Collection interface allows
objects from any schema to be stored, indexed, and accessed in a unified manner.
          </p>
          <p>
With Collections, you store, query, and retrieve data based on the very schema
and type of data you seek. Here are a few example data objects from a variety of
common schemas that entities may write and access via a user's Hub:
          </p>
          <p>
<strong>Locate any offers a user might want to share with apps</strong> (http://schema.org/Offer)
          </p>
          <pre class="example">
{
  "@context": "https://schema.identity.foundation/0.1",
  "@type": "ObjectQueryRequest",
  "iss": "did:foo:123abc",
  "sub": "did:bar:456def",
  "aud": "did:baz:789ghi",
  "query": {
      "interface": "Collections",
      "context": "http://schema.org",
      "type": "Offer",
  }
}
          </pre>
        </section>
        <section>
          <h4>
 Services
          </h4>
          <p>
 Services offer a means to surface custom service calls an identity wishes to
 expose publicly or in an access-limited fashion. Services should not require
 the Hub host to directly execute code the service calls describe; service
 descriptions should link to a URI where execution takes place.
          </p>
          <p>
Performing a <code>Request</code> request to the base <code>Services</code>
interface will respond with an object that contains an entry for every service
description the requesting entity is permitted to access.
          </p>
          <pre class="example">
// request
{
  "iss": "did:foo:123abc",
  "sub": "did:bar:456def",
  "aud": "did:baz:789ghi",
  "@context": "https://schema.identity.foundation/0.1",
  "@type": "ServicesRequest",
}

// response
{
  "@context": "https://schema.identity.foundation/0.1",
  "@type": "ServicesResponse",
  developer_message: "optional message",
  services: [{
    // Open API service descriptors
  }]
}
          </pre>
          <p>
All definitions shall conform to the <a href="https://github.com/OAI/OpenAPI-Specification">Open API descriptor format</a>.
          </p></p>
      </section>
    </section>
agropper commented 3 years ago

The point of this section seems to be:

"Identity Hubs are a formulation of Encrypted Datastores .... wherein a developer of an application or service stores the data for a Subject with that Subject, in their Identity Hub, instead of in a traditional centralized location owned and controlled by a third-party."

It is understood that the "application or service" is processing data rather than just storing it, otherwise, the EDV section would be sufficient. As a processor, the (application or) service has access to data in the clear. (I'm ignoring secure multi-party computation and such in this response. Please raise a separate issue if you want to deal with that.)

The issue, from the perspective of the Subject (of the "identity hub") is not where the storage is hosted but who controls access the contents of the storage. For example, as a Subject, I might self-host storage in my closet and expose that to to various Services to use but it would hardly protect me form these services having their way with my data.

This issue is nicely discussed from the perspective of service providers in the context of the Uniform Law Commission Committee (where I'm an 'observer') Main Street Privacy Coalition to: Collection and Use of Personally Identifiable Data Drafting Committee where they explain the need for separation of concerns and effective regulation of platforms that combine multiple concerns.

Therefore, I strongly object to the proposed section on Identity Hubs as part of the spec and seek to have it removed entirely.

If it's within scope to specify a separate layer above the EDV access authorization, that layer must preserve the separation of concerns between policy decision points and the policy enforcement point. If we choose to specify this upper layer, we should also consider how auditing will be done in order to detect security breaches at the policy enforcement point if the PEP controls access to data in the clear.

My proposed solution to 'processing data in the clear' would be to define the authorization protocol that gives substantially all control to an agent of the Subject as the policy decision point. I don't know if that means a separate layer above the EDV or a parallel specification at the same layer as the EDV. That's related to VCs - zCaps / OCap a Discussion.

csuwildcat commented 3 years ago

@agropper "It is understood that the "application or service" is processing data rather than just storing it, otherwise, the EDV section would be sufficient. As a processor, the (application or) service has access to data in the clear." - the vast majority of data that will flow through Identity Hubs (down into EDVs) is encrypted, and the Hub has no access to it. The only data that is send to a Hub in plaintext is an intended-public object like a publicly declared Tweet, that the Hub has no special access to - it's literally just hosting data like you would when you post blog entries for everyone to read.

Obviously it would be folly to remove the most useful, high value portion of the specification that represents what the vast majority of application and services developers need to begin creating apps that rely on the user's own personal datastore as their backing data storage mechanism. For this reason, I can't even fathom why we would make the spec relatively useless to the vast majority of valuable use cases by removing it.

agropper commented 3 years ago

@csuwildcat I maintain that the management of intended-public objects should have absolutely nothing to do with the Confidential Storage specification.

However, I would support adding a section about implementation guidance somewhere that tells people how they might set up a searchable directory of intended-public objects and what, if anything about that directory relates to the normative aspects of the Confidential Storage spec.

For example, when we decide to specify the authorization protocol for access to an EDV, we could mention that we recommend that directory creators support the same protocol so that the same data controller can interact with both EDV and intended public-object directories.