Willow-powered Store class

[sgwilym]

This PR introduces a new, Willow-powered Store class, replacing Replica. This class is used to create, update, and retrieve Documents, which also have also been updated.

The new name reflects the reality that most of the time, peers will not hold identical copies of a share’s data, and instead have their own view of the share based on their capabilities and who they’ve synced with.

This branch relies on a specific version of willow-js to be installed locally. It will be updated soon to depend on publicly available releases so that it is easier to try yourself.

Store

Example usage:

const store = new Store("+gardening.bhynoq5vqfpysmi2i7zilhdnynfsuq5wddus5sfgce24z53a2f6da");

await store.set({
    identity: "@suzy.b3kxcquuxuckzqcovqhtk32ncj6aiixk46zg6pkfocdkhpst4selq",
    path: ['greetings', 'earth'],
    payload: new TextEncoder().encode("Hello world!"),
});

const doc = await store.get({
    identity: "@suzy.b3kxcquuxuckzqcovqhtk32ncj6aiixk46zg6pkfocdkhpst4selq",
    path: ['greetings', 'earth'],
});

Store has the following methods:

• set - Create or update a document for a given identity and path.
• clear - Update an existing document with an empty payload.
• get - Retrieve a document using an identity and path.
• documents - Iterate through all of a store’s documents.
• latestDocAtPath - Retrieve the newest document at a given path.
• documentsAtPath - Iterate through all documents at a given path.
• queryDocs - Iterate through docs selected by a query (see below).
• queryPaths - Iterate through paths of documents selected by a query.
• queryIdentities - Iterate through identities of documents selected by a query.

The new Query type looks like this:

type Query = {
    /** A path all documents must be prefixed by. */
    pathPrefix?: string[];
    /** The identity which wrote the document. */
    identity?: IdentityAddress;
    /** The earliest point at which a document was written, in microseconds. */
    timestampGte?: bigint;
    /** The latest  point at which a document was written, in microseconds. */
    timestampLt?: bigint;
    /** The maximum number of documents to be returned. */
    limit?: number;
    /** The maximum cumulative size of all returned documents' payloads. */
    maxSize?: bigint;
    /** The order in which documents will be returned. Uses `path` by default.
    *
    * - `path` - path first, then timestamp, then identity.
    * - `timestamp` - timestamp first, then identity, then path.
    * - `identity` - identity first, then path, then timestamp.
    */
    order?: "path" | "identity" | "timestamp";
    /** Whether to return results in descending order. `false` by default. */
    descending?: boolean;
};

The Store class is also an EventTarget and emits events which can be listened to using addEventListener and other standard APIs. The events which can be listened to are as follows:

• documentset
• documentingest
• payloadingest
• documentremove
• payloadremove

Relevant data for each can be found on the event’s detail property.

Document

The Document data type has been redesigned:

type Document = {
    /** The share this document belongs to. */
    share: ShareAddress;
    /** The identity associated with this document. */
    identity: IdentityAddress;
    /** The path this document corresponds to. */
    path: Path;
    /** When the document was written. */
    timestamp: bigint;
    /** The size of the document's payload in bytes. */
    size: bigint;
    /** The SHA-256 digest of the payload, encoded as a base 32 string. */
    digest: Base32String;
    /** The identity used to authorise this document's creation. */
    signedBy: IdentityAddress;
    /** The data associated with this document. */
    payload: Payload | undefined;
};

ShareAddress is a string e.g. +gardening.bhynoq5vqfpysmi2i7zilhdnynfsuq5wddus5sfgce24z53a2f6da. IdentityAddress is a string e.g. @suzy.b3kxcquuxuckzqcovqhtk32ncj6aiixk46zg6pkfocdkhpst4selq. Path is an alias for string[]. Payload is an object with length, getBytes, and getStream properties (see willow-js).

The / character is no longer permitted to be used in a path.

Misc

• Crypto has been renamed CryptoES so as not to collide with the global Crypto type.
• Moved all the pre-Willow source into src-old.