Unify UCAN API endpoints

[gobengo]

Motivation:

• w3up phase 2 p0 - https://hackmd.io/i7kliu8NS3Gu3upnfn0WAA?both

To Do

• [x] #331
• [x] try out upload-client connecting to http endpoint of access-api worker https://github.com/gobengo/monorepo/blob/997cf85472a2f887d5a3ac60dd1127fa6f98de5a/experiments/w3protocol-test/src/w3protocol.test.ts#L16
• [x] https://github.com/web3-storage/w3protocol/issues/359

Problem

• w3protocol clients should only need one ucanto connection
• Right now, clients of our ucanto APIs need to keep track of at least two different http URLs that they use to create their ucanto connections
• and they need to know which capabilities can be handled by which ucanto connections: e.g. in w3up-client
• and the client must be sure to use ucanto connection URLs that are from the same 'environment' (e.g. staging) or else bad things could happen
  • We want clients to be able to make a single connection to web3.storage and issue all w3protocol capabilities to that connection and have it work

Solution

• there should be a single ucanto connection URL that can handle all access/, store/, etc. invocations
• this ucanto connection api hides implementation details of separate 'access-api' and 'upload-api' services behind the scenes, and creates a layer of indirection. this is like the api gateway pattern
• the ucanto api gateway
  • non-goals
  • validating UCAN signatures
  • goals
  • ucan routing
    • introspect requests to extract a ucanto http invocation
    • based on value of can of invocation, determine backend ucanto service that can handle it, i.e. upload-api or access-api
    • forward request to backend service, and proxy response from backend service back to the client requesting the ucanto api gateway
  • implement ucan routing handler in a way that it can be composed into other projects, e.g. as a request handler function that be called from another node server, lambda, or cf worker

Nice to Have

• did:web:web3.storage did document links to the ucanto api via service endpoint
  • Then there is a way of doing FollowYourNose from just the UCAN aud of did:web:web3.storage to the URL you should send ucanto invocations to:
    1. resolve the did doc of your aud - for did:web:web3.storage, this means GET https://web3.storage/.well-known/did.json
    2. look in "service" property for a service endpoint of type ucanto
    3. create a ucanto http connection to url from the ucanto service endpoint
    4. send the ucan invocation on that connection (e.g. by HTTP POST)
    5. didn't have to keep track of any info (serviceURL, servicePrincipal) other than the aud, which you need anyway to create the ucan invocation.

Decisions

✅ What should the URL be of the unified ucanto endpoint?

Status

• 2021-01-10: after decision 2 leading to just putting unified functionality in existing access-api (not creating new worker), decide to not even make a new URL. Just re-use access-api URLs like access.web3.storage

Ideas

• bengo
  • production: https://ucan.web3.storage
  • staging: https://ucan-staging.web3.storage
    • rationale: similar to current 'api.web3.storage' and 'api-staging.web3.storage'. flat subdomain space simpler than hierarchy.
  • rationale
    • brief discussion with @Gozala . We both would prefer using:
    • domain web3.storage - but it's currently served by cloudflare pages (repo) in a way we don't want to risk changing (yet)
    • domain api.web3.storage - but it's currently served by cloudflare workers (repo) in a way we don't want to risk changing (yet)
    • let's get something working/functional at ucan.web3.storage. After that milestone of functionality and seeing access/upload invocations go through a single endpoint, then we can revisit final domain, e.g.
    • an option but tbd: use the w3s-ucanto-gateway handler inside the @web3-storage/api handler. This is kinda nice as it enhances our existing/legacy API URL with ability to handle ucanto http requests by proxing to the new microservices.

✅ Should the unified UCAN API endpoint respond to websocket connections?

Status

• 2023-01-10: decided SC1 with @Gozala

Context

• when testing a prototype with access-client, it failed because access-client expects to be able to establish a websocket connection to its api endpoint (i.e. access-api)
  • cases
  • when agent does createspace - e.g. when the access-client dev cli does w3access create-space, when it will connect to websocket to find out when the email has been confirmed

Problem

• access-client as-is can't use the unified endpoint unless
  • the endpoint responds to the websocket connection request
  • we update access-client to determine the websocket URL using some new algorithm
• it's not clear how to do a 'generic' websocket handler in the unified ucan api endpoint because there are no specs for how a ucanto server should handle websocket connections. i.e. access-api websocket protocol is access-api specific.

Considerations

• access-client/agent opens Websocket specifying 'validate-ws' protocol/topic https://github.com/web3-storage/w3protocol/blob/main/packages/access-client/src/agent.js#L488

Solutions Considered

1. 😅 don't have api-gateway at all....
   • decided on 2023-01-10 with @Gozala
   • just have the unified api endpoint be access-api
     • i.e. add code to access-api/service so the underlying service can handle invocations to upload-api by re-executing them via a client connection over http channel
     • implications
     • also decided: no need in this issue to make upload-api able to handle access-api caps. We will only make it so access-api can handle all caps. (and there will be no api-gateway that can handle all caps, which is what I had started)
2. api-gateway responds to incoming websocket connections by redirecting to access-api worker
   • pros
     • simple to implement
   • cons
     • doesn't consider what to do for other services.
     • but we can decide that later when we actually have the problem of multiple services that can communicate over websocket
3. access-client can discover separate access-api websocket url from unified api did document
   • e.g. a service endpoint at #access-api and/or service of type w3protocol.web3.storage/services/access-api
   • ucanto invocations could still go through unified api, but websocket would have to connect to the other URL.
   • pros
     • unified api endpoint is simpler because no websocket logic
     • websocket connection url can still be discovered by following nose from the service DID (DID -> did doc -> services(typ
   • cons
     • access-client needs to know that there are separate URLs for websocket vs ucanto-over-http, which is kinda worse than its status quo of connecting to same access-api URL for both
4. api-gateway handles incoming websocket connections by proxying to upstream access-api
   • pros
     • fully encapsulates upstream services
     • if we unify the websocket handling protocol, then specific upstream services may not need to implement one directly, and the unified one can aggregate events from across services
   • cons
     • unclear how to generically handle websocket connections (i.e. what protocols do we expect? how do we decide to proxy to access-api vs upload-api et al?)

[gobengo]

this is a little related to https://github.com/web3-storage/w3protocol/issues/182

[gobengo]

status today will address feedback on this PR adding initial functionality: https://github.com/web3-storage/w3protocol/pull/334#pullrequestreview-1244943356

[gobengo]

status you can now issue store/list invocations against access-api

test here. store/add invocations are also proxied ok to upload-api handlers, and currently get a result from this in upload-api

[gobengo]

status I will update it to use new ucanto functionality to do the proxying without issuing a new delegation - https://github.com/web3-storage/w3protocol/issues/383

[gobengo]

status In general after above work, you should be able to issue ucan invocations only against the deployed access-api. If they are store/* or upload/* invocations, they will be proxied to be handled by the upload-api running on aws.

This was the gist of how @Gozala and I decided to do it (especially at the time in January).