Alternative Bitcoin backend(s)

[darosior]

The only Bitcoin backend the first release will have, and that we'll support in the near future, is Bitcoin Core using a watchonly wallet there. Other software tend to (ab)use the Electrum protocol and other kinds of backends (explorer APIs for instance). We might want to support other Bitcoin backends than Bitcoin Core.

For instance i can think of those as being desirable:

• Electrum (but we need to introduce Tor, and/or TLS... Meh)
• Maybe a Bitcoin Core backend that does not require a watchonly wallet (scantxoutset?)

Thoughts? Is there interest from users for prioritising this feature?

[darosior]

Looks like https://github.com/bitcoindevkit/rust-electrum-client could be a reasonable choice for an Electrum backend. Depends on their supported MSRV of course.

Or maybe even BDK? It looks like a lot of their features have been made optional, so we could just use it as the Electrum backend without pulling more dependencies than their own code. That sounds reasonable to both not reinvent the wheel and concentrate efforts there should we need improvements or have bug fixes. Provided there is a decent MSRV too. See https://github.com/bitcoindevkit/bdk.

[darosior]

Another alternative. Sexier but that requires more work, is supporting BIP158 using Nakamoto. https://github.com/cloudhead/nakamoto

[darosior]

Looks like BDK's MSRV is 1.46. Which works for us. But i think they are going for 1.48 now, which is still fine.

[vincenzopalazzo]

Another alternative. Sexier but that requires more work, is supporting BIP158 using Nakamoto. https://github.com/cloudhead/nakamoto

I like to have this alternative, and I will put some more love on it in the near future, so if you think to reconsidered it we may want to talk some day

[darosior]

Awesome! Feel free to share design decisions or a WIP branch here if you'd like to have feedback before posting the PR.

-------- Original Message -------- On May 14, 2023, 11:40 AM, Vincenzo Palazzo wrote:

Another alternative. Sexier but that requires more work, is supporting BIP158 using Nakamoto. https://github.com/cloudhead/nakamoto

I like to have this alternative, and I will put some more love on it in the near future, so if you think to reconsidered it we may want to talk some day

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[vincenzopalazzo]

I am in touch with @cloudhead, so I plan to work upstream all my changes, for now, are all inside different PR https://github.com/cloudhead/nakamoto/pulls and I arrived at a proof of concept with a core lightning plugin.

Would it be cool to have a list of things that you need, one of my problems currently is estimating the fee like bitcoind, but I need some free time to think about it harder

[darosior]

To estimate fees you'd need to relay transactions. I though Nakamoto was only about BIP158.

[vincenzopalazzo]

Nakamoto has a way to keep track of the fee from the block processing message, I also exposed an API with this commit https://github.com/cloudhead/nakamoto/pull/139/commits/acc70aaa79a302898eabadcad5a11a540201cfd4, but it is a monkey way.

I need to think more hard about how to do this.

To estimate fees you'd need to relay transactions.

I had no real-world experience with this, so I think I need to make a more deep reading of the estimation fee of Bitcoin core, but I guess we could use in nakamoto the BIP133 as Neutrino does

I though Nakamoto was only about BIP158.

What do you mean? The BIP158 is the definition of the Golomb-Coded Sets, but nakamoto is offering an light way way to access the bitcoin network like neutrino does

[siwatanejo]

but nakamoto is offering an light way way to access the bitcoin network like neutrino does

But this way of accessing the blockchain still requires more data downloading than another kind of light client, for example: electrum client. Right?

I need to think more hard about how to do this.

Did you come up with something in the past months? If not, I'm thinking that querying some wellknown online services (e.g. mempool blockexplorer) for this might not be ideal, but a temporary solution until a better one can be thought of?

[vincenzopalazzo]

Did you come up with something in the past months?

Yes, I had some private problem to solve, but I have a couple of branch that implements the nakamoto support, so I will make PoC by the end of the month

[darosior]

But this way of accessing the blockchain still requires more data downloading than another kind of light client, for example: electrum client. Right?

Yes.

If not, I'm thinking that querying some wellknown online services (e.g. mempool blockexplorer) for this might not be ideal, but a temporary solution until a better one can be thought of?

Sure, a PR to implement Electrum support would be welcome. Note it's not as trivial a task as you might think though.

On a related note, we (Wizardsardine) have been working on a new service. There is discussions about integrating this service to Liana here, which would allow one to not need bitcoind and access other services provided by this backend.

[siwatanejo]

Note it's not as trivial a task as you might think though.

Let me guess, the thing that inspired you to say this is: server selection (algo or UI), right?

[darosior]

More like adapting all our Bitcoin logic to support both bitcoind and Electrum: https://github.com/wizardsardine/liana/tree/master/src/bitcoin.

[vincenzopalazzo]

In here there is a logic to use nakamoto https://github.com/vincenzopalazzo/liana/tree/macros/nakamoto

I need just to spend some time to understand how track the transaction and avoid liana crashing

[nondiremanuel]

We will try to address the Electrum part (inserting it into v7 for this reason). I paste here the possible approach proposed by @darosior, to be discussed. We should divide the activity in 3/4 parts:

1) Track transactions in our own database, not in bitcoind's watchonly wallet (remove the last dependency on bitcoind) 2) [optional] Make the API with the "Bitcoin Backend" more Electrum-friendly as right now it's very bitcoind-focused 3) Implement a "Bitcoin Backend" which uses BDK to sync against Electrum with minimal dependencies 4) Introduce it in the GUI as a configuration

In order to foster the discussion, I tag @jp1ac4 (in case you wanted to share some thoughts on the approach). Thanks!

[jp1ac4]

We should divide the activity in 3/4 parts:

Quick update: I've created #1180 to address Step 1 and am currently working on Step 3 (as part of which I will probably identify and make changes towards Step 2).