search

Tribler / tribler

Privacy enhanced BitTorrent client with P2P content discovery
https://www.tribler.org
GNU General Public License v3.0
4.87k stars 451 forks source link

A Mobile and Generic Asset Exchange using Liquidity Pooling [group 1] #5988

Closed devos50 closed 3 years ago

devos50 commented 3 years ago

This ticket contains the notes for the liquidity pooling project (#5944)

A Mobile and Generic Asset Exchange using Liquidity Pooling

Ever since the introduction of Bitcoin, there has been a proliferation of blockchain-based currencies and applications. Nowadays, the majority of blockchain-based applications orient around decentralized finance (DeFi), an experimental form of finance that enables value transfer and exchange directly between participants, without intermediaries. The number of different DeFi application is quickly growing. Many of these DeFi applications are deployed using smart contract logic, e.g., on Ethereum.

Currently, decentralized exchanges (DEXes) make up a significant part of the DeFi landscape. DEXes enable the trade between different pairs of digital assets, without risks. Some DEXes facilitate trade by having traders create buy and sell orders and automatically execute orders when a trading opportunity arises. Newer DEXes like Uniswap use a different approach to trade assets. In this approach, there is a pool for every pair of assets, and users can deposit assets in these pools. Such users are called Liquidity Providers and are rewarded with special 'share' tokens. Traders can then exchange asset A for B by depositing asset A in the pool, in exchange for asset B.

Existing DEXes using liquidity pooling only allow the trade of assets that are locked to the ecosystem in which the DEX operates. For example, Uniswap can only exchange Ethereum-based assets, significantly limiting the applicability of this approach. Recently, we have introduced an approach to enable generic asset trading using traditional order books (we will share the paper with you at the start of the project). This approach enables the trade of any asset, managed by any ecosystem. Risks are manageable by splitting up a trade into a series of smaller, intermediate payments.

The goal of this project is to engineer a basic and generic DEX, based on Liquidity Pooling. All operations and asset transfers will be recorded on TrustChain, our lightweight distributed ledger. This approach makes it possible to detect fraudulent behavior, specifically stealing assets. You will integrate this application in our Superapp, which already has several decentralized applications integrated and deployed (e.g., for social networking, music streaming, and federated machine learning). Your final demo will show a simple asset exchange between two real cryptocurrencies, e.g., Bitcoin and Ethereum tokens. For simplicity, you may assume that the liquidity pool will be managed by a trusted identity in the network.

anatomy

References:

(image source: https://uniswap.org/docs/v2/protocol-overview/how-uniswap-works/)

synctext commented 3 years ago

Sprint ToDo ideas.

devos50 commented 3 years ago

Meeting notes 12-02-2021:

thversfelt commented 3 years ago

TODO for monday 15-02-2020, 09:00:

daandeheij commented 3 years ago

Useful short introduction to Bitcoin blockchain: https://www.youtube.com/watch?v=Lx9zgZCMqXE

thversfelt commented 3 years ago

Compilation instructions:

devos50 commented 3 years ago

To get a feeling for how liquidity pools work, you can play a bit around on the Uniswap website.

daandeheij commented 3 years ago

Short detailed paper on Constant Market Maker model implementation:

thversfelt commented 3 years ago

Possible solution: modified luxury communism to incorporate an additional wallet on a different blockchain (not BTC)

daandeheij commented 3 years ago

Possible solution: modified luxury communism to incorporate an additional wallet on a different blockchain (not BTC)

Also:

thversfelt commented 3 years ago

Possible solution: modified luxury communism to incorporate an additional wallet on a different blockchain (not BTC)

Also:

* modify luxury communism in order for traders to propose exchanges with the liquidity pool

* change entry fee logic to liquidity pool entry logic (record proportion of assests added)

* create logic for liquidity providers exiting liquidity pool and retrieving funds

Another idea:

daandeheij commented 3 years ago

Work done 17-02-2021:

Questions that came up during work meeting:

Code meeting planned for 18-02-2021 (9:30):

daandeheij commented 3 years ago

Work 18-02-2021:

devos50 commented 3 years ago

Meeting notes 19-02-2021:

Focus of this sprint: enable universal support for different crypto wallets in the luxury communism app. Prepare the fundamental interfaces for your liquidity pool ideas and give a small code demo next week.

thversfelt commented 3 years ago

Progress:

devos50 commented 3 years ago

Meeting notes 25-02-2021:

Next sprint:

devos50 commented 3 years ago

Idea for next sprint: get taproot (multi-)signatures operational in Kotlin code (also see https://github.com/Tribler/tribler/issues/5984). To achieve this, you have to compile the Bitcoin reference implementation for Android.

thversfelt commented 3 years ago

Progress update 02/03/2021:

thversfelt commented 3 years ago

Progress update 03/03/2021:

Demo: There are two providers, who create personal BTC wallets as well as a multi-sig wallet requiring 2/2 signatures to redeem from. First, provider 1 creates a 0.02 BTC provide transaction to the multi-sig wallet. Then, provider 2 creates a 0.01 BTC redeem transaction and signs it with his own key. Provider 1 signs the redeem transaction with his own key, and finally provider 2 broadcasts the transaction to receive the 0.01 BTC from the multi-sig wallet.

multi-sig-demo

Note: the demo is run on a single client on BTC REGNET with artificial delays and free funds from a BTC faucet. Next step might be to explore using trustchain for communicating multi-sig wallet addresses, as well as provide/redeem transaction info. That way it can be more easily tested on multiple clients using TESTNET. As requested, we should also be working on getting currencii (Luxury Communism) to perform what is currently implemented, as to avoid code duplication. It was preferable for now to get a hang of programming a bare minimum implemtation of a multi-sig wallet, in order to better understand currencii's implementation.

devos50 commented 3 years ago

Next sprint:

Technical specifications can be found here.

thversfelt commented 3 years ago

Taproot implementation idea:

daandeheij commented 3 years ago

Meeting 09/03/2021:

thversfelt commented 3 years ago

Progress 10/03/21

devos50 commented 3 years ago

Meeting notes 12-03-21:

Next sprint: get Ethereum up and running for Android. Ethereum provides pre-built binaries for Android. This gives you a .aar file that you can link against a .apk with Android Studio.

daandeheij commented 3 years ago

Notes 16-03-21:

daandeheij commented 3 years ago

Notes 17-03-21:

EmielBoss commented 3 years ago

Notes 18-03-21:

thversfelt commented 3 years ago

Notes 18-03-21:

* Generated Java wrapper for our compiled Ethereum contract `MultiSigWallet.sol` with Web3j

* Deployed this multisig wallet Java wrapper

* Created half a demo in which we create wallets tap Ether, but because faucets have a limit and we cannot seem to tap any, we haven't been able to fully develop the transactions yet

* Last attempt at getting taproot to work, but it turned out to be too much of a hassle

Compiling android for getting Taproot to work now has a guide from a member of the bounty project (https://github.com/Tribler/tribler/issues/5986#issuecomment-799589124).

thversfelt commented 3 years ago

The following demo shows 2 liquidity providers creating personal ETH wallets. Then, provider 1 creates an ETH multi-sig using both providers' addresses. Currently, faucets have blocked my IP to redeem ETH for the providers, so providing and redeeming funds is a bit of a TODO, but should be pretty straightforward.

Demo2

devos50 commented 3 years ago

Meeting notes 19-03-2021:

Next sprint:

daandeheij commented 3 years ago

Notes 24/03/2021

thversfelt commented 3 years ago

Progress 24/03/2021:

daandeheij commented 3 years ago

Notes 25-03-2021

thversfelt commented 3 years ago

Progress 23/03/2021:

devos50 commented 3 years ago

Meeting notes 26-03-2021:

For next sprint, please dedicate 2 team members on getting TrustChain+ BTC multi-sig up and running and one member on Android geth.

devos50 commented 3 years ago

I got a geth node up and running on Android, with relative ease. Relevant logcat output:

2021-03-26 10:38:07.088 8188-8237/nl.tudelft.gethtest E/GoLog: INFO [03-26|09:38:07.088] New local node record                    seq=11 id=757b66d288dd5650 ip=127.0.0.1 udp=33703 tcp=33703
2021-03-26 10:38:07.133 8188-8240/nl.tudelft.gethtest E/GoLog: INFO [03-26|09:38:07.126] Started P2P networking                   self=enode://dd2040638419d8bed55a12be6a753813cb026d1ca5a3f673345ee3b15ad5fab370c9cdfed0735305c2bb7c8bbfe911c5973b00f8e7d7554f4d85ac90924c170f@127.0.0.1:33703
2021-03-26 10:38:07.133 8188-8240/nl.tudelft.gethtest E/GoLog: WARN [03-26|09:38:07.127] Light client mode is an experimental feature 
...
2021-03-26 10:38:42.519 8188-8241/nl.tudelft.gethtest E/GoLog: INFO [03-26|09:38:42.518] Looking for peers                        peercount=0 tried=1 static=0
2021-03-26 10:39:07.694 8188-8242/nl.tudelft.gethtest E/GoLog: INFO [03-26|09:39:07.692] Looking for peers                        peercount=0 tried=1 static=0
2021-03-26 10:39:08.497 8188-8241/nl.tudelft.gethtest E/GoLog: INFO [03-26|09:39:08.497] Block synchronisation started 
2021-03-26 10:39:16.267 8188-8237/nl.tudelft.gethtest E/GoLog: INFO [03-26|09:39:16.266] Imported new block headers               count=192 elapsed=319.8ms number=11969279 hash="96d396…ce0567" age=3w1d6h
2021-03-26 10:39:16.790 8188-8239/nl.tudelft.gethtest E/GoLog: INFO [03-26|09:39:16.790] Imported new block headers               count=192 elapsed=291.541ms number=11969471 hash="2dc3f4…153f1c" age=3w1d5h
2021-03-26 10:39:17.375 8188-8234/nl.tudelft.gethtest E/GoLog: INFO [03-26|09:39:17.375] Imported new block headers               count=192 elapsed=315.084ms number=11969663 hash="91226e…b035d2" age=3w1d4h
2021-03-26 10:39:17.907 8188-8239/nl.tudelft.gethtest E/GoLog: INFO [03-26|09:39:17.906] Imported new block headers               count=192 elapsed=310.665ms number=11969855 hash="3b62c4…033d63" age=3w1d4h
2021-03-26 10:39:18.224 8188-8234/nl.tudelft.gethtest E/GoLog: INFO [03-26|09:39:18.224] Imported new block headers               count=192 elapsed=316.848ms number=11970047 hash="c8bef4…04ec86" age=3w1d3h
2021-03-26 10:39:18.890 8188-8242/nl.tudelft.gethtest E/GoLog: INFO [03-26|09:39:18.890] Looking for peers                        peercount=1 tried=1 static=0

I started with a fresh app, and imported the .aar file using the GUI: File -> new -> new module -> import .aar. I then had an additional module that I renamed to geth. Adding implementation project(':geth') to the dependencies section in my app's build.gradle makes the app compile successfully.

Starting the geth instance (I did this in the onCreate method in my main activity):

val nc = NodeConfig()
val node: Node = Geth.newNode("$filesDir/.ethNode", nc)
node.start()
val gethNode = GethNodeHolder.getInstance()
gethNode.node = node

This will by default run a light node that will only synchronize block headers. I think it is possible to enable full sync (also see this issue) but that might be too demanding for Android phones (although high-end devices might be able to do it).

A few suggestions:

thversfelt commented 3 years ago

Progress 30/03/2021:

To-do tomorrow (31/03/2021):

thversfelt commented 3 years ago

Mandatory screenshot: 😋 image

EmielBoss commented 3 years ago

Progress 30/03/2021 (more verbose, for Daan to understand where I'm at):

daandeheij commented 3 years ago

Current issues Thijs is running into (and looking at together now):

daandeheij commented 3 years ago

Progress 30/03/2021 (more verbose, for Daan to understand where I'm at):

daandeheij commented 3 years ago

Some documentation on trying to deploy contract with Geth

Great work @EmielBoss on compiling the new contract and diving into it! And also @thversfelt great work on the liquidity pool! I will document here what I tried today. The docs on Geth are extremely limited, and especially for the android implementation. Overall, the android implementation feels limited and immature.

Still don't know how in the world to deploy/bind a contract.

  1. First I looked into the constructors of MultiSigWallet:

The contract binding (MultiSigWallet.java) has two constructors and a deploy function, but I don't know how to call those. One constructor asks for Address and EthClient arguments, while the other asks for a BoundContract

Those constructors are there such that you can create an instance of MultiSigWallet that is bound to a contract that is already on the Ethereum blockchain (so that you can interact with it). Therefore, it asks for an Address; which is the location of the contract on the blockchain, or alternatively for a BoundContract instance (which will pretty much also point to the location of the contract). So unfortunately, we cannot use those constructors now. They could be helpful though, if we can first sort of manually publish the contract.

  1. Naturally, I then looked at the deploy method of MultiSigWallet:

Its parameters are as follows (type + name):

So there is no issue at all with the last three parameters! We have a client thanks to geth, and I managed to get the owner addresses into an Addresses instance. _required simply states the number of owners that need to approve the transaction.

  1. The revenge of TransactOpts:

    However..., TransactOpts is a bit more of a mystery. It can be easily made using the Geth.newTransactOpts() method, and after that you can set various fields, such as gasPrice and fromAddress. I tried to set these fields. It would be easier if geth could generate this TransactOpts for us, which seems to be possible with NewKeyedTransactOpts. However, I can't find this method in the Android geth version.... As one of its field is a Signer, I even created a KeystoreSigner class, which I use to create a Signer instance which I then put in the TransactOpts. However, when I run it with all fields, I get a hex not valid error. Debugging is a bit difficult, since I can't step into the geth code lines.

The links that Martijn posted in his latest comment might be very useful; I haven't looked at that thoroughly.

They are quite useful! I was already aware of the sendTransaction method. But especially the second link to some test code was a really nice find.

So what are the options now?

I personally would prefer tomorrow to try the second option, but not spend too much time on it. I think the last option is also very feasible, and would be cool to try. But overall, I feel that we have gone very far to try to incorporate geth into our project. We have already spent 10+ (maybe 20+) hours going through geth code. I feel that it would be a waste of resources to continue incorporating it into our project, especially since we now know that there is virtually no documentation. This poses a real risk for our project later on. If we need to change something regarding geth later, then it will again most likely take a lot of resources due to no docs.

Some last words on web3j and infura

I understand that it is undesirable to depend upon a (external) gateway to the Ethereum network such as infura. However, it is still my opinion that our previous set-up with web3j and infura is sufficient for now. If later on more mature Ethereum clients become available for Android, then we can easily incorporate them into the project, as web3j supports rpc. So then we can focus again on developing the liquidity pool, and it will be future proof due to its design. I argue that it is in the best interest of this project to choose a library that is more extensively documented and developed. Let's get back to focusing our resources on the liquidity pool, instead of getting lost in a detail.

thversfelt commented 3 years ago

About our work: I have informed group 2 about our app refactor and BTC multi-sig implementation on our intercommunication issue (https://github.com/Tribler/tribler/issues/6001#issuecomment-811921753). I will quote what I told them here, just to keep a good overview of our work:

BTC multi-sig implementation update (branch: https://github.com/chrislemaire/trustchain-superapp/tree/multi-sig-wallets-implementation):

* Refactored our BTC multi-sig demo into a more Object-Oriented format

* Refactored liquidity pool app code to make place for BTC multi-sig

* Implemented BTC multi-sig wallet creation into the app

* Implemented BTC multis-sig depositing, and withdrawing into the app

The implementation is made on top of what code group 2 has presented last week. This means it only has partial trustchain integration. It required quite some refactoring, so it will look a bit different than what was presented to us. I do think that in its current state, group 2 should be able to use the current BTC implementation, to be able to complete their BTC-EURotoken demo.

Meanwhile, group 1 (us), will create a seperate fragmant in which we will contain and implement our BTC-ETH demo. This should allow us to work more independently from now on. Of course, please communicate any questions regarding BTC multi-sig with us.

I am looking forward to our meeting tomorrow.

daandeheij commented 3 years ago

Update on geth integration:

On branch geth-server, web3j now connects to Rinkeby network using (independent) geth node.

Instructions:

Work for next week

The method web3j.ethGetBalance is not working yet with this geth node. Perhaps some api options are currently missing.

thversfelt commented 3 years ago

Hey guys, I found a really useful guide for implementing Geth on Android with deploying smart contracts: https://freesoft.dev/program/108526336

It shows how to:

Will try getting this to work, but it looks very promising.

devos50 commented 3 years ago

Meeting notes 06-04-2021:

Deliverables for next week Friday:

daandeheij commented 3 years ago

Progress update yesterday and today (07/04/2021):

This is a joint write-up of Daan and @thversfelt

Work done:

What problem we have run into now:

Locally, transactions and contract deployments are successful. However, they don't seem to be broadcasted to the Etherereum (Rinkeby) network. We have about 4 connected peers at any given time. Locally, we have an instance of our contract, and the geth node also confirms that the contract has been deployed (including hashes). However, because there is no bound contract (to the contract's address), we can't interact with the (multisignature) contract.

What we tried:

Possible causes:

Likely no causes (because we debugged):

What may be worth trying:

Perhaps @devos50 would like to look into this problem. Any help would be greatly appreciated. Please see this commit

devos50 commented 3 years ago

@daandeheij That's very unexpected. Are you sure that the light node has fully catched up with the network (also see this SO post)? Also, are you sure that broadcasting is part of the deploy_contract function? It might be a separate method.

Instead of running Geth on android, running Geth on windows (light mode) and connecting to it through Android: not broadcasting our contract.

Is the Windows light node not broadcasting the contract or are you not seeing it on the Android side?

As a last resort, please try to run a full node and see if you can issue your transaction successfully. This should be a small configuration change right?

devos50 commented 3 years ago

Also, you might want to join the Ethereum discord/IRC channel and ask their help 👍

daandeheij commented 3 years ago

I just came across an interesting post on stackexchange. The user states that he used to be able to broadcast transactions successfully using a light node previously, but that lately his transactions aren't broadcasted anymore.

I will follow that post closely. Hopefully there will be some answers to it. In the meantime, I will first run a full node on my laptop to see whether it works there. That would confirm that our code is working, and would rule out a bug there.

Edit: Right now I'm running a Geth node on Windows with syncmode=fast. It's syncing right now, and it's almost at age=1y. Hopefully it finishes synchronization tonight!

daandeheij commented 3 years ago

08-04-2021

Current problem