Design naming / version evolution scheme for WNS

dboreham commented 4 years ago

Having moved to immutable record naming in WNS, we need to figure out how to support concepts like "the latest version of xxxx" and friendly names for entities such as bots in WNS.

ashwinphatak commented 4 years ago

Re: bot record:

RB: ok — so is this record the record for a bot factory INSTANCE or a bot artifact?
what happens if two people register the same bot? (currently one would overwrite the other)
lots of issues here that we need to resolve before we bake this (already v2)
(also, this doesn’t sound like an “access key”)
call it “topic”?

ashwinphatak commented 4 years ago

Naming/Versioning/Ownership Issues

How do we find the latest (or semver based) version of some artifact?
How do we avoid users/developers having to deal with CIDs in config files (e.g. to refer to a particular protocol version in bot.yml).
How do we handle multiple versions of the same bot/pad (e.g. fetching all versions would look messy in the current LP members dropdown)?
How do we prevent creation of multiple CIDs with the same (type, name, version) combination, e.g. (wrn:bot, wireline.io/chess, 0.1.0), by changing other attributes in the record.
How do we prevent other users from creating newer versions of a record they don't own?

Proposed Solution

queryRecords GQL API -> support server semantics for version field in queries.
- If version not provided OR version === 'latest', return only the latest version for each (type, name) combination.
- If semver string, return highest matching version for each (type, name) combination.
- If version === '*', return all versions
Auto-created unique index in WNS that maps a reference/ref string of the form wrn:<type>:<name>#<version>, to a particular record/CID. Also, a resolveRecords([ref]) GQL API that returns records given ref strings.

Examples
- wrn:protocol:wireline.io/chess always points to the latest version, updated automatically when a record is published.
- wrn:protocol:wireline.io/chess#0.1.0 points to version 0.1.0 of the record, created automatically when the record is published.
CLI commands
- wire wns resolve wrn:bot:wireline.io/chess (version is implicit, fetches the latest record)
- wire wns resolve wrn:bot:wireline.io/chess#^2.3.0 (semver based, fetches record for highest matching version)
- wire wns resolve wrn:bot:wireline.io/chess#0.1.0 (fetches record for specific version)
Checks on ref (unique index) when creating records.
- Prevent creation of record with (type, name, version) combination if ref already exists. Disallows records with same (type, name, version) being created just by changing some other metadata, resulting in a different CID.
- Prevent publishing of newer version if owners don't match latest version. Ensures a single dev/group/org is in control of the (type, name) combination.
Refs make it easier for humans to refer to a particular artifact version, e.g. in config files. The tooling MUST throw an error if the mapping does not exist in WNS.

# bot.yml
id: wrn:bot:wireline.io/shogibot
name:ShogiBot
version: 0.1.0
protocol:
  type: wrn:reference
  ref: wrn:protocol:wireline.io/chess#0.1.0
  # id: QmZBjSuaLhRuFVi7ti215678v26C1V4vWcrRaXmb7s65qi

Note: Use either ref OR id. If ref, version should be explicit (i.e. no semver or 'latest').

ashwinphatak commented 4 years ago

Artifacts vs Services

Artifacts: Protocol, Pad, Bot - represent code artifacts/metadata.
Services: Compute (e.g. BotFactory) - represent always on/running services that can be invoked to perform some task.

Compute services and bots.

Bot instances don't need to be in WNS. Typically just peers in a party.
Bots are scoped to a user or party, created by a compute service, destroyed/spun down after use.
Bots need an address (way to reach them) on the P2P network (e.g. a swarm topic).
Compute services (e.g. BF like entities) ARE registered on the WNS and could spawn any bot by downloading bot artifacts from WNS. In the most limiting case, they're capable of spawning instances of a single bot artifact.
Compute services will declare a protocol (e.g. BotFactory protocol) that indicates they're able to spawn bots.
Note: Bot addressing and live instances are separate concerns (like FaaS), i.e. bot might be restarted/rehydrated by the BotFactory on access.
Note: Rename accessKey to topic. Topic could change and should be part of the mutable metadata in the record.
Note: Bot Factories are services and registered as such (wrn:service?). Capabilities are denoted by protocols they support, specified in record attributes.

CLI commands.

wire protocol register (currently: wire wns publish --filename="protocol.yml")
wire pad register
wire bot register

wire service register
wire service run

AFDudley commented 4 years ago

Prevent publishing of newer version if owners don't match latest version. Ensures a single dev/group/org is in control of the (type, name) combination.

This feels a bit like a kludge for "chess" being a first class name. I feel like people/orgs should be reserving names and then they are responsible for putting what they want under that name, i'm not sure i'm completely following along, so maybe I'm confused?

dboreham commented 4 years ago

@AFDudley that's the idea. This work focused specifically on versions, but there's another task to take care of hierarchical namespace with subtree authority/access control. It turns out they're the same thing pretty much.

ashwinphatak commented 4 years ago

Namespace/access control issue: https://github.com/wirelineio/wns/issues/32

wirelineio / wns