livepeer / go-tools

Utility packages used across Livepeer Go repositories.
0 stars 2 forks source link

Add w3 CLI script for w3s driver #17

Closed leszko closed 1 year ago

leszko commented 1 year ago

fix https://github.com/livepeer/catalyst/issues/383 kudos to @red-0ne for starting this work in his PoC PR: https://github.com/livepeer/catalyst-api/pull/387

Context In the w3s driver, we use w3 CLI to interact with web3.storage. With the current implementation, we use the credentials created by w3 CLI (stored in the local FS). In other words, currently, we don't use the credentials provided by the user, which is not correct and not what we need for web3.storage integration.

What we want is to pass the credentials (UCAN private key configured with the catalyst-api flag and the delegation proof passed by the user) into w3 CLI and use them to interact with the web3.storage service. There are, however, two issues:

  1. w3 CLI does not provide a way to import the private key (though it allows importing deletion proofs)
  2. w3 CLI stores its credentials (and its state) in the local FS; at the same time we need multi-tenancy because different users need to use w3 with different proofs.

Solution This PR adds a JS script that can be used in the same way as w3 CLI for our use cases, but it allows passing private key and poof as env variables. So, instead of executing w3 can store add <car-file>, we will execute the following command.

$ W3_PRINCIPAL_KEY=<key> W3_DELEGATION_PROOF=<proof> node w3.mjs can store add <car-file>

Additional comments:

Alternative solutions

1. Implement this as a feature in w3 CLI

The best for us would be if this is actually implemented in the w3 CLI. I've sent a PR to their repo (https://github.com/web3-storage/w3cli/pull/42). I'm not sure it'll be merged, but let's try it.

2. Import credentials into w3 CLI

We could write a script (@red-0ne already wrote it here) and then use w3 profiles for multi-tenancy.

Nevertheless, I think that solution is worse because it would still mean maintaining our own "import" script and additionally:

3. Fork w3 CLI

We could fork the w3cli repo and maintain our own version, but I'm not sure it's worth it.

4. Move this script to a separate repo

Instead of keeping this script in livepeer/go-tools (or livepeer/catalyst / livepeer/catalyst-api), we could create a separate repo and store it as a complete JS module (with package.json, etc.).

leszko commented 1 year ago

LGTM either way. We can even keep this script on this separate branch until we figure this out better.

re: distributing to catalyst, have you given a try on tools like pkg? These should create a hopefully distributable binary to run the node script. Might be simpler than adding node support in catalyst container (if we don't have that yet)

I tried pkg, but actually encountered some issues while packing these dependencies. So, we'll probably need to stick with installing nodejs.