Why favour libbitcoin servers over SPV?

[JustinDrake]

As I understand, the current OpenBazaar node implementation is built around libbitcoin to gather information from the Bitcoin network. Right now there is exactly one libbitcoin server used by the OpenBazaar network. This server is hardcoded in the source code, is owned by OB1, and is flaky. The plan for the future is to have a consortium of libbitcoin servers (presumably run by OB1 and other entities). OpenBazaar nodes will pick at random from the pool of libbitcoin servers for redundancy. Is this understanding technically correct and fair?

Now the standard decentralised way for nodes to gather information from the Bitcoin network with a light footprint is to use SPV. According to this brief discussion, the two reasons for favouring libbitcoin servers over SPV are:

1. "the moderator can't look up a transaction and get the inputs if it's SPV"
2. there is no good Python SPV client

I don't understand the first argument. Anyone, including a moderator, can look up a transaction using SPV.

I also don't understand the second argument. OpenBazaar can use libraries written in languages other than Python.

Why favour libbitcoin servers over SPV?

[drwasho]

Right now there is exactly one libbitcoin server used by the OpenBazaar network.

No. There are multiple libbitcoin servers that we run and that we share with Airbitz.

This server is hardcoded in the source code, is owned by OB1, and is flaky.

I agree it's flaky in testnet mode. Seems to be ok for mainnet.

The plan for the future is to have a consortium of libbitcoin servers (presumably run by OB1 and other entities). OpenBazaar nodes will pick at random from the pool of libbitcoin servers for redundancy.

That's already in place, AFAIK the only thing is that we need to finish setting up is how nodes access the libbitcoin servers we have and that we share so that if one goes down, it automatically tries another one. Lastly, the user can manually change the location of the libbitcoin servers in their settings, using their own if they so wish.

Anyone, including a moderator, can look up a transaction using SPV.

@cpacia can correct me if I'm wrong, but that's not how SPV works. In SPV mode you're only doing 2 things: 1) validating and relaying transactions, and 2) keeping a record of the block headers. Because you don't have a copy of the blockchain or the UTXO, so you can't check any 'balances'. This means you have to query a full node to get this information.

This brings us right back to why Libbitcoin was chosen in the first place. It's a federated full node built to serve this information to anyone, and anyone is free to spin one up.

OpenBazaar can use libraries written in languages other than Python

Yes and we'll have to evaluate whether porting an existing library in another language is easier or harder than further developing Chris' repository written in Python.

I think it would be worth doing the latter and actually making a kickass SPV client to keep a copy of the UTXO set. Overall seeing hundreds of thousands or millions of OpenBazaar nodes that are also SPV clients will significantly strengthen Bitcoin (especially when you throw in fraud proofs) as well as allowing us to drop Libbitcoin.

But we can't do everything. We have limited resources to commit to such a thing. So if you can marshal an effort to do this, the closer it will become a reality.

[cpacia]

If we we're to use a SPV library it would probably need to be bitcoinj as that's the gold standard. We'd have to try to get it working with jython because that sounds better than building an api into a jar (which would work as well).

The main issue is as washington says, you can't look up balances in SPV mode. You can only listen for transactions coming off the wire. As it relates to the moderator getting the inputs, they could probably just be sent in the dispute message. Not sure if the moderator needs verification the coins are in the chain.

There's also an issue with the ratings as they are kind of designed to look up the address and make sure a transaction took place. We don't do that atm out of scalability concerns.

So SPV could possibly work. Something to look into going forward.

[JustinDrake]

@drwasho A few replies. Please correct me where I am wrong.

that's not how SPV works. In SPV mode you're only doing 2 things: 1) validating and relaying transactions, and 2) keeping a record of the block headers [...] you can't check any 'balances'

I think that is incorrect. There are SPV wallets (such as Breadwallet and Mycelium) and they can check balances. To check balances an SPV wallet requests Merkle proofs from the Bitcoin network and verifies them based on the longest chain of block headers it has seen. To preserve privacy when looking up transactions Bloom filters are used.

making a kickass SPV client to keep a copy of the UTXO set

A client that keeps a copy of the UTXO set is called a pruned node and is no longer considered an SPV client. The UTXO set is over 1.1 GiB and is growing over time. I do not think this makes sense in the context of OpenBazaar. (Do we want nodes to download gigabytes of data?)

We have limited resources

Using an off-the-shelf SVP wallet is still an option.

[drwasho]

To check balances an SPV wallet requests Merkle proofs from the Bitcoin network and verifies them based on the longest chain of block headers it has seen.

That's exactly what I said though. The SPV has to query another full node for balance details.

The UTXO set is over 1.1 GiB and is growing over time. I do not think this makes sense in the context of OpenBazaar.

Yes that's a good point. I don't it's reasonable to ask our users to hold an ever-expanding UTXO set. Perhaps a custom client that the user could opt-into, SPV vs pruned mode, would be cool.

Using an off-the-shelf SVP wallet is still an option.

Like @cpacia bitcoinj's SPV client is the best candidate... we just have to evaluate the resource burden for users if we go with this approach. That said, I'd still prefer our own in Python at some point.

[JustinDrake]

Ok, cool. The SPV client can query whoever has the data. It can be a full node, a pruned node, or even something adhoc like an OpenBazaar node that already has the information. Querying data is a basic action for nodes in a P2P network. I don't see that as a justification for libbitcoin's client-server model.

[cpacia]

It's a bit of a misnomer. SPV wallets really don't query anything. They can only listen for transactions as they come off the wire and calculate their balances that way. To the extent they get a "balance" for their addresses at startup, they have to request that the remote peer load every single block, filter it, then send it to them. So a "query" in this sense requires either downloading/re-downloading every single filtered block from the wallet creation date going forward.

If you wanted to get the balance of a random address, you would need to re-download the entire (filtered) blockchain. That's where server based wallets have an advantage. They index the blockchain in a way that normal full nodes don't so they can serve individual address balances without needing to serve up all filtered blocks.