`Unauthorized: Claim {"can":"provider/add"} is not authorized` when trying to register space - when juggling prod/staging W3UP_SERVICE_DID

[gobengo]

Motivation:

• This is a nuanced bug I ran into when trying to test another bugfix released to staging. To do that, I want to use w3cli to do some stuff with a space. But in order to test the change on staging and not prod, I ran w3cli with the env vars

  W3UP_SERVICE_DID=did:web:staging.web3.storage
  W3UP_SERVICE_URL=https://staging.up.web3.storage

  But when I did this for a new space, I was unable to register it using w3cli

How to Reproduce Issue

Reproduction Steps

(this is what works for me while reporting this. Due to the nature of it, there may be further setup required to reproduce. e.g. you probably need to have done a w3 authorize once before without the env vars to reproduce from scratch)

In terminal

1. export W3UP_SERVICE_DID='did:web:staging.web3.storage'; export W3UP_SERVICE_URL='https://staging.up.web3.storage'
2. w3 authorize bengo@dag.house
   • then look in email and click link
3. Note: w3 up ~/Pictures/sharkDAO/GOPR0788.JPG would fail right now due to having no storage provider. So we need to do a w3 space register
4. w3 space register -e bengo@dag.house

Expected Behavior

the space is registered, and then I can upload stuff to the space using w3 up ~/Pictures/sharkDAO/GOPR0788.JPG

Actual Behavior

bengo@bengo ~ ⚡  w3 space register -e bengo@dag.house  
{
  name: 'Unauthorized',
  stack: 'Unauthorized: Claim {"can":"provider/add"} is not authorized\n' +
    '  - Capability {"can":"provider/add","with":"did:mailto:dag.house:bengo","nb":{"provider":"did:web:staging.web3.storage","consumer":"did:key:z6Mknd3mqhEdr7i2ePbF5GhLHjAMi4NnjCTt6zj3RSjs1JLM"}} is not authorized because:\n' +
    "    - Capability can not be (self) issued by 'did:key:z6MkuyxsFdrZSZZHG2KNDuFFKgbEHkT3PT3KWqNJdQqFLeMB'\n" +
    '    - Capability can not be derived from prf:bafyreida733uonbaspd2lhjdgwtuepacodbpt4rgd53pxfbtmcrbfdmlpq because:\n' +
    "      - Unable to resolve 'did:mailto:dag.house:bengo' key\n" +
    '    - Capability can not be derived from prf:bafyreihzbwqllxswxjxsb3ocat43lcqef3kpoenyyvcsid7ei3qxlicnje because:\n' +
    "      - Unable to resolve 'did:web:web3.storage' key\n" +
    '    at claim (/node_modules/@ucanto/validator/src/lib.js:308:12)\n' +
    '    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n' +
    '    at Object.add (/node_modules/@ucanto/server/src/handler.js:61:27)\n' +
    '    at invoke2 (/node_modules/@ucanto/server/src/server.js:145:23)\n' +
    '    at async Promise.all (index 0)\n' +
    '    at execute (/node_modules/@ucanto/server/src/server.js:104:5)\n' +
    '    at ucanInvocationRouter (/upload-api/functions/ucan-invocation-router.js:204:20)\n' +
    '    at Runtime.handler (/node_modules/src/awslambda.ts:308:1)',
  message: 'Claim {"can":"provider/add"} is not authorized\n' +
    '  - Capability {"can":"provider/add","with":"did:mailto:dag.house:bengo","nb":{"provider":"did:web:staging.web3.storage","consumer":"did:key:z6Mknd3mqhEdr7i2ePbF5GhLHjAMi4NnjCTt6zj3RSjs1JLM"}} is not authorized because:\n' +
    "    - Capability can not be (self) issued by 'did:key:z6MkuyxsFdrZSZZHG2KNDuFFKgbEHkT3PT3KWqNJdQqFLeMB'\n" +
    '    - Capability can not be derived from prf:bafyreida733uonbaspd2lhjdgwtuepacodbpt4rgd53pxfbtmcrbfdmlpq because:\n' +
    "      - Unable to resolve 'did:mailto:dag.house:bengo' key\n" +
    '    - Capability can not be derived from prf:bafyreihzbwqllxswxjxsb3ocat43lcqef3kpoenyyvcsid7ei3qxlicnje because:\n' +
    "      - Unable to resolve 'did:web:web3.storage' key"
}

Interesting things about this error

• last line Unable to resolve 'did:web:web3.storage' key". remember that because this is all running with W3UP_SERVICE_URL=https://staging.up.web3.storage, we/w3cli shouldn't expect proofs issued by did:web:web3.storage to be accepted by the similar-software-but-otherwise-separate did:web:staging.web3.storage

### Tasks
- [ ] https://github.com/web3-storage/w3up/pull/1047
- [x] update w3cli to use fix in w3up#1047
- [x] release w3cli
- [x] test fix using released w3cli

[gobengo]

Note:

• @web3-storage/access agent gets session proofs without worrying much about the issue of those sessions and whether they are issued by the other end of the connection. that's probably how we can test/fix https://github.com/web3-storage/w3up/blob/main/packages/access-client/src/agent.js#L263

[gobengo]

When testing locally, I witnessed the provider/add invocation going out with only one session proof, i.e. only one proof of ucan/attest. And that proof was issued by did:web:web3.storage not the did:web:staging.web3.storage that corresponds to the W3UPSERVICE env vars I'm using.

So I think we can fix this like

• create test scenario for this case where a single access agent is re-used with several ucan/attest issuers.
• then test for each of the 2 auth providers used by the agent, the agent can be used to send a provider/add invocation that should always contain a ucan/attest issued by the agent the invocation is sent to
  • expect this to fail right now
  • then make a fix for it
• because w3cli uses an access agent instance, expect the above to fix this issue in w3cli once w3cli incorporates the fix in access Agent

[gobengo]

I've been debugging this this morning, and found out that part of the issue is in @web3-storage/access Agent #delegations method https://github.com/web3-storage/w3up/blob/main/packages/access-client/src/agent.js#L186

When that is called internally to find UCANs to send to the provider/add invocation, it finds a delegation in the agent data for the relevant key/capability, so it calls _caps.delete(cap) there. But it shouldn't. Because the delegation it found and determined to authorize the action may not actually authorize it if it requires a session proof issued by a different ID (e.g. did:web:staging.web3.storage instead of did:web:web3.storage)

[gobengo]

I tested w3cli@4.6.0 and was unable to reproduce this issue, which leads me to believe this issue is fixed! 🎈