Specify interface and architecture for wallet and blockchain data modules

[chimp1984]

_This is a Bisq Network project. Please familiarize yourself with the project management process._

Description

Research solutions for the wallet and blockchain data modules, provide a proposed solution and specify interface and architecture based on the targeted protocols and blockchains.

Rationale

For the multi-protocol / multi-blockchain project we need to support different blockchains and wallets. Beside that there should be also flexibility for users to make a trade off between best secruity/privacy (full node) vs. low resource requirements (light node).

Criteria for delivery

• A document summarizing the research (e.g. which options have been investigated/are available) is provided
• If there is a prototype, the link to the repo (no need to have that inside Bisq, no code quality constrainsts as it is only for prototyping).
• The proposed solution
• Basic RFCs for specification (not needed to be final but to have a level which covers the core aspects sufficinetly).
• The scope of different solutions should include:
  • BTC full node (e.g. Bitcoin Core) for current Bisq trade protocol use case
  • BTC light node (e.g. Electrum) for current Bisq trade protocol use case
  • BTC on Lightning network for an off-chain security model (e.g. reputation based). LN-BTC only used for transfer.
  • Liquid, LTC, Zcash will likely work similar like the BTC version
  • Monero java RPC lib can be considered for an off-chain security model (e.g. reputation based). XMR only used for transfer.
  • Monero should be considered in the context of re-using the Farcaster or Comit code base (in Rust) for atomic swaps
  • Ethereum should be considered as potential candidate
  • RSK should be considered as potential candidate

Measures of success

The provided results are considered sufficient to be used as base for refining the specification.

Risks

• No solution found
• No sufficient progress
• Quality of investigation insufficient
• Quality level of RFCs insufficient
• Solution does not fit goals or constraints of overall project

Tasks

• [ ] Investigate and research options
• [ ] Design architecture
• [ ] Verify with a prototype the ideas if helpful
• [ ] Write up proposed solution
• [ ] Write RFCs

Estimates

This investigation should be scoped from effort and time. I think 1 month for 1 contributor should not be exceeded. According to that the contributor should post their expected compensation.

Notes

We cannot ship all those different modules in the app, so we need a dynamic on-demand loading concept (see https://github.com/bisq-network/projects/issues/53).

We do not want to take liability for the wallets, so goal is to have only the wallet features required for Bisq trades and keep everything else outside. The wallet binaries will not be part of the Bisq installation but the user has to install them (or point to already installed ones). Reference implementations of wallets should be preferred.

Security is very important and there should be some option that the user can limit risks (e.g. max. allowed transfers per time period). An isolated wallet is preferred over the model that the user connects to their standard wallet which might contain large funds and carries higher risk. Hardware wallet integration should be considered, thought details about integration would be out of scope of that project.

@oscarguindzberg did an investigation about alternatives to BitcoinJ 2 years ago. The scope of the summary matches the expectation of that project. It also can be used as base for checking for light wallet alternatives for Bitcoin. Back then Neutrino was not considered production ready and Electrum was the best choice. https://github.com/bisq-network/bisq/issues/1062#issuecomment-446755367

The Farcaster project RFCs: https://github.com/farcaster-project/RFCs We do not require that level of detail and qualtiy at that stage, but their RCFs can be taken as inspiration and template.

This high level architecture can serve as inspiration as well: https://github.com/farcaster-project/RFCs/blob/master/03-farcaster-architecture.md They have similar challenges as they want to support different blockchains.

There will be some overlap with #53, #54. Those 3 related projects should interact to share findings and report on preliminary results.

[ghost]

I will devote up to 1 month on this and my estimate for cost would not exceed $8000. If my deliverable is not up to standards the DAO can decline my compensation request.

I will deliver progress reports twice a week posted to this thread so that:

• (a) it keeps me focused on results,
• (b) if I get stuck or go down a wrong path suggestions can be made.

Plan of work:

• Identify actions that a generic blockchain/wallet module has to fulfil => document.
• Write a document detailing what existing Bisq would need to change to adapt using the generic wallet module => this work would also help define the scope of a generic wallet module.
• Look at the potential blockchains/wallets to be supported and how they can be abstracted by a generic module interface => document each.
• Produce a functional prototype of the most promising solution which shows how the various use cases are handled.
• Adapt the prototype to a different wallet/chain backend, thus showing how it fulfils the requirement.
• Any lessons learnt from the prototyping can be applied back to the design doc/RFCs.
• Investigate what potential issues if a wallet/technology does not support spending chained unconfirmed transactions.

My intention is that prototypes would feature heavily as supporting evidence to the final deliverables.

Link to: Notes;
Link to: Work log;

[ghost]

Update: https://github.com/chimp1984/misq/discussions/5#discussioncomment-748405 & also the two documents linked above.

[ghost]

• Updated doc with latest notes & conclusions. https://gist.github.com/jmacxx/587ac9a59d7d060b4f9fdf37fa9cc8ef
• Added a second prototype running an example 2-of-2 multisig trade https://github.com/jmacxx/misq_prj52_WIP.
• Also added basic electrum wallet RPC to prototype.

[chimp1984]

Do you think the scope of the project is completed? From what I see it seems so.

[ghost]

What remains, from my point of view:

• IMO there needs to be the concept of capability in the interface so that Misq knows if a provider can be used for what kind of trading methodologies (not all chains and not all wallet providers can satisfy the advanced use cases).
• Produce a design. (a generic interface handling different implementations while being able to satisfy various use cases: normal transfers; unsecured trading; escrow; timelocks). The prototypes can be considered a rough design, but I thought you had asked for a RFC document. I would expect to take 2 days creating it.
• You mentioned focus on bitcoind and electrum; however I think we also need to provide a third option: BitcoinJ exposed thru the same generic interface.
• If using electrum for trading with escrow we'll need something that helps construct the transactions (i.e. getting the scripts/keys/addresses all sorted out). A library - currently we use BitcoinJ for that.

Lets discuss in tomorrow's meeting.

[chimp1984]

Yes agree an interface for providing the implementations capability would be useful. Though I think we should focus on wallets which support all main use cases. Some blockchains like Monero will be excluded for some protocols due lack of some features (timelock), so that will an additional restriction we have to keep in mind.

I see a RFC document optional at that level, but you can start a draft for it if you think it helps.

Re Electrum/BitcoinJ: So the Electrum API does not support all the required functionality we need? That would be bad. Maybe we can extend it? I have my concerns to use BitcoinJ again due all the issues. Also having the wallet in the same process has higher security risks. For constructing a transaction we could use different approaches as its now in BitcoinJ (where we can construct it on OpCode level). For instance to use a byte representation of the script and replacing the keys which are marked as placeholders (Comit seems to use such an approach).

[ghost]

• Added a third and fourth prototypes using electrum to do 2-of-2 multisig trade deposit https://github.com/jmacxx/misq_prj52_WIP.

Spent a lot of time fighting to get electrum to be able to sign a combined transaction between Alice & Bob for funding a multisig trade. Electrum's transaction building routines are a bit weird and nowhere near as good as bitcoin core. Had to resort to deugging electrum code to find out how it works and eventually come up with something barely acceptable. Electrum 4 has rudimentary PSBT support and so alice and bob can each separately sign the same transaction then the PSBT can be combined into a finialized transaction. I did not try the payout part but it should work just the same as the deposit part - they are both two entities signing the same Tx.

https://github.com/jmacxx/misq_prj52_WIP/blob/master/misqWalletRpc4/README.md

I'll leave the Electrum demo at this stage and move on to LND.

[ghost]

• Updated prototypes to include a LND wallet interface and added Capability enum to the wallet interface so clients can figure out what the chosen wallet can do. https://github.com/jmacxx/misq_prj52_WIP/tree/master/misqWalletRpc5

Currently only referencing the mainchain functionality. Lightning functionality might conceptually be a sub-interface if possible. Especially as the LND API for lightning is large.

I've spent a month on this it should be concluded by now. Will shift gears back to legacy Bisq work.

[chimp1984]

Close as completed