Closed leszko closed 1 year ago
LGTM either way. We can even keep this script on this separate branch until we figure this out better.
re: distributing to catalyst, have you given a try on tools like
pkg
? These should create a hopefully distributable binary to run the node script. Might be simpler than adding node support in catalyst container (if we don't have that yet)
I tried pkg
, but actually encountered some issues while packing these dependencies. So, we'll probably need to stick with installing nodejs.
fix https://github.com/livepeer/catalyst/issues/383 kudos to @red-0ne for starting this work in his PoC PR: https://github.com/livepeer/catalyst-api/pull/387
Context In the w3s driver, we use w3 CLI to interact with web3.storage. With the current implementation, we use the credentials created by
w3
CLI (stored in the local FS). In other words, currently, we don't use the credentials provided by the user, which is not correct and not what we need for web3.storage integration.What we want is to pass the credentials (UCAN private key configured with the catalyst-api flag and the delegation proof passed by the user) into
w3
CLI and use them to interact with the web3.storage service. There are, however, two issues:w3
CLI does not provide a way to import the private key (though it allows importing deletion proofs)w3
CLI stores its credentials (and its state) in the local FS; at the same time we need multi-tenancy because different users need to use w3 with different proofs.Solution This PR adds a JS script that can be used in the same way as
w3
CLI for our use cases, but it allows passing private key and poof as env variables. So, instead of executingw3 can store add <car-file>
, we will execute the following command.Additional comments:
Dockerfile
to install node andnpm
dependencieslivepeer/catalyst
orlivepeer/catalyst-api
Alternative solutions
1. Implement this as a feature in
w3
CLIThe best for us would be if this is actually implemented in the
w3
CLI. I've sent a PR to their repo (https://github.com/web3-storage/w3cli/pull/42). I'm not sure it'll be merged, but let's try it.2. Import credentials into
w3
CLIWe could write a script (@red-0ne already wrote it here) and then use w3 profiles for multi-tenancy.
Nevertheless, I think that solution is worse because it would still mean maintaining our own "import" script and additionally:
3. Fork
w3
CLIWe could fork the w3cli repo and maintain our own version, but I'm not sure it's worth it.
4. Move this script to a separate repo
Instead of keeping this script in
livepeer/go-tools
(orlivepeer/catalyst
/livepeer/catalyst-api
), we could create a separate repo and store it as a complete JS module (withpackage.json
, etc.).