Alter devenv to Allow Leather Wallet to Connect

[kyranjamie]

Leather depends on the Mempool.space Bitcoin API, rather than directly on a node. This is an instance of Blockstream's Esplora API.

For compatibility with the sbtc dev regtest environment (without a long, costly refactor of the wallet), we'll need an instance of this API to point to. If this can be added to the dev environment easily enough, it'll be trivial our side to add a network directly, like

cc/ @markmhendrickson

[EmbeddedAndroid]

We have an electrum api (which also is packaged with esplora), is that what is needed?

To rephrase, what services from esplora do you folks need?

[EmbeddedAndroid]

We have looked into this, and here are the issues: https://github.com/hirosystems/clarinet/issues/1140

[EmbeddedAndroid]

The bitcoin explorer currently in devenv has an api: https://github.com/janoside/btc-rpc-explorer/blob/master/docs/api.js

Is that something that could be used?

[friedger]

It is only a question on providing the url? Where in the wallet?

[radicleart]

It is only a question on providing the url? Where in the wallet?

in the screen where you add a new network? Will it also need support for the devnet:devnet credentials on the url ?

[AshtonStephens]

I'm changing the name of this ticket to "Alter devenv to Allow Leather Wallet to Connect" because esplora isn't an option since it expects to interact with stacks nodes in a way that stacks nodes don't support.

[friedger]

Who are esplora and stacks node related? I'm confused.

[AshtonStephens]

Ah, okay, my understanding was incorrect but the conclusion, as I understand it, is the same:

Per Tyler's observation here https://github.com/hirosystems/clarinet/issues/1140#issuecomment-1711816528. Esplora is a bundle of a bunch of services, not all of which we need. But more importantly, Esplora only supports cookie auth and not password auth.

Here is where my understanding begins to deteriorate; it sounds like if we want to use both local esplora and a stacks node we need them to both to support the same authentication type. Because stacks nodes don't support cookies and Esplora only supports cookies we need to alter one or the other... or just not use Esplora.

@EmbeddedAndroid would you be able to explain why they both need the same auth type or correct my explanation?

---

It looks like the leather wallet likely doesn't even need esplora; it might just need the electrs API. A version of electrs API is already included in the devenv.

It sounds like the hold up with getting the Leather Wallet extension to work locally is the lack of an esplora API, but looking at the Esplora GitHub it seems that a the api is actually a specific fork of electrs. Which means, more strictly, that leather wallet likely only requires the electrs API.

The esplora team forked the original electrs repo a while ago, so there may be some API differences between the electrs that we’ve pulled in here and the one that Leather needs, but it would be great if we could identify whether this API has interfaces Leather needs.

• Esplora GitHub
• Esplora’s Electrs GitHub
• Original Electrs GitHub

If you find out that this electrs API doesn't have some functionality that the leather wallet needs we could investigate including the forked version of electrs API as well or instead of the existing one.

I’ve looked through the Leather Wallet GH and the only note I see of esplora is in bitcoin-transaction.model.ts. So I have a few questions.

1. Where does leather get the API endpoint for esplora and where can that be configured?
2. What esplora specific interactions does the leather wallet make outside of API calls that are defined within electrs (if any)?

When devenv spins up the API becomes available at localhost port 60401. What can we do to help determine what options are available?

[EmbeddedAndroid]

Ah, okay, my understanding was incorrect but the conclusion, as I understand it, is the same:

Per Tyler's observation here hirosystems/clarinet#1140 (comment). Esplora is a bundle of a bunch of services, not all of which we need. But more importantly, Esplora only supports cookie auth and not password auth.

Here is where my understanding begins to deteriorate; it sounds like if we want to use both local esplora and a stacks node we need them to both to support the same authentication type. Because stacks nodes don't support cookies and Esplora only supports cookies we need to alter one or the other... or just not use Esplora.

@EmbeddedAndroid would you be able to explain why they both need the same auth type or correct my explanation?

Esplora supports cookie authentication for the bitcoin RPC requests.

As an example, when the container starts, it generates a cookie. https://github.com/Blockstream/esplora/blob/5601a25ef6a7ffa935f324397ce03adad35db47a/contrib/liquid-mainnet-explorer.conf.in#L6

All services inside esplora: https://github.com/Blockstream/esplora/blob/master/run.sh use this cookie file to authenticate with the bitcoin RPC since it's local to the container file system. External services that require access to the P2P/RPC bitcoin interface must also use this cookie.

In order to use esplora, you would need to generate the cookie file, and share it with all containers needed access to bitcoind.

Which means configs like this will no longer work - https://github.com/stacks-network/sbtc/blob/main/romeo/asset-contract/deployments/default.devnet-plan.yaml#L6 also https://github.com/stacks-network/sbtc/blob/main/devenv/stacks/docker/Config.toml#L90 since it's uses password auth. Anywhere in devenv where a RPC password is set, would need to use the cookie file from esplora.

The other option is hacking the esplora container to use password auth for all services. I tried this but got frustrated trying to unwind everything included.

[AshtonStephens]

@kyranjamie can you point us to the interactions where the Leather Wallet needs Esplora? Again, we're hopeful that the existing electr API interface has what's needed.

[kyranjamie]

Didn't realise Esplora is a fork of electrs. If the schema is the same as Esplora/Mempool.space, great, we can use that.

Most bitcoin-related API calls are defined here.

I'm trying to spin up the devenv now to test against the port 60401 (though it's taking some time 💤)

[AshtonStephens]

I'm trying to spin up the devenv now to test against the port 60401 (though it's taking some time 💤)

@kyranjamie thanks for the update! Is there anywhere where we can track this work and provide any insight on our end?

[kyranjamie]

I understand that the RESP API is added in Blockstream's fork of electrs. So romanz/electrs has nothing Leather can consume in its current state.

Tried firing requests at it but seems like an RPC API? 🤷🏼‍♂️

``` electrs | [2023-10-02T20:21:57.196Z WARN electrs::electrum] invalid JSON ("POST / HTTP/1.1"): expected value at line 1 column 1 electrs | [2023-10-02T20:21:57.196Z WARN electrs::electrum] invalid JSON ("Host: localhost:60401"): expected value at line 1 column 1 electrs | [2023-10-02T20:21:57.196Z WARN electrs::electrum] invalid JSON ("Accept-Encoding: gzip, deflate"): expected value at line 1 column 1 electrs | [2023-10-02T20:21:57.196Z WARN electrs::electrum] invalid JSON ("Accept: */*"): expected value at line 1 column 1 electrs | [2023-10-02T20:21:57.196Z WARN electrs::electrum] invalid JSON ("Connection: keep-alive"): expected value at line 1 column 1 electrs | [2023-10-02T20:21:57.196Z WARN electrs::electrum] invalid JSON ("Content-Length: 0"): expected value at line 1 column 1 electrs | [2023-10-02T20:21:57.196Z WARN electrs::electrum] invalid JSON ("User-Agent: HTTPie/3.2.2"): expected value at line 1 column 1 electrs | [2023-10-02T20:21:57.196Z WARN electrs::electrum] invalid JSON (""): EOF while parsing a value at line 1 column 0 ```

Any chance https://github.com/Blockstream/electrs can be used in its place?

[AshtonStephens]

Okay, I'll give it a shot. Worst case we may be able to have both.

[friedger]

For the cli and the binary we currently used tcp client only.

[AshtonStephens]

@kyranjamie what can I do locally to verify whether leather will work with the api? How do I configure the endpoint that leather is using to communicate with the local electrs container?

[kyranjamie]

Currently the form doesn't allow you to select regtest. I'm thinking we'll add a hardcoded sBTC option you can select from the list.

I'll put this in a test build, but I'll need to know on what port you'll put the Blockstream esplora on?

[friedger]

Port 60401, I guess

[AshtonStephens]

Heads up @kyranjamie, getting the esplora electrs up is proving difficult. I've started investigating whether we can help support changing the api that leather wallet uses.

While we don't have the same mempool.space api supported, we do have an api available via the block-explorer with very similar functionality that's hosted on localhost port 3002.

https://github.com/janoside/btc-rpc-explorer/blob/master/docs/api.js#L11C86-L11C86

For example, the leather wallet uses this to get recommended fees:

• https://mempool.space/api/v1/fees/recommended Where it could instead use this for dev net.
• http://127.0.0.1:3002/api/mempool/fees
• (the mainnet version) https://bitcoinexplorer.org/api/mempool/fees

Which would respond with this:

{"nextBlock":50,"30min":50,"60min":50,"1day":50}

I think it would be fairly easy to have this altered with a leather configuration variable for devnet in places like this

[AshtonStephens]

The Esplora version of the electrs API is proving very difficult to spin up. Not only are a number of its dependencies very old, but we need to use the bullseye debian image to pull in the right version of librocksdb-dev, at which point the electrs code won't compile because of errors in cargo 1.47, which is the latest cargo version available on the bullseye image.

Even using debian:bullseye@sha256:4d6ab716de467aad58e91b1b720f0badd7478847ec7a18f66027d0f8a329a43c from the esplora docker image isn't enough.

---

I'm going to pivot to attempting to get the mempool api github running on its own so that leather can hook into it.

After that, @kyranjamie has agreed to look into altering the leather wallet code to match with the bitcoin explorer API which is already in the devenv environment.

It'll be good to have the bitcoin explorer API supported anyway.