Capitalisk / lisk-v3-dex-adapter

LDEX adapter for Lisk v3 and its forks.
1 stars 0 forks source link

Requirements #1

Open jondubois opened 3 years ago

jondubois commented 3 years ago

The lisk-v3-dex-adapter Node.js module needs to do the following things:

jondubois commented 3 years ago

I think the best way to test it would be to launch a Lisk v3 node; you can connect to alphanet or testnet. You should find instructions online how to do it and get help from Lisk community - Then the v3 node will create a rocksdb database directory (it stores it on the file system; kind of like SQLite but it uses multiple files so you need to point the client to the directory path).

On my node, it stores the rocksdb data inside ~/.lisk/lisk-core/data/blockchain.db/

So then you need to use these client modules: https://www.npmjs.com/package/rocksdb and https://www.npmjs.com/package/levelup together to connect to that rocksdb database folder and read data from it.

You can see how lisk node uses the rocksdb client here: https://github.com/LiskHQ/lisk-sdk/blob/2db69cdd52b2ea8461525a34c388619921481c64/elements/lisk-db/src/kv_store.ts#L55

RocksDB is a key-value database so there are no queries, you can only find items by key - This is why we need to synch and copy the data to postgres.

To sync, you should iterate over each block (for each height) and for each one, copy all the transactions in that block into postgres and for each of those transactions, see which accounts are affected (sender and recipient) and also copy them from rocksdb to postgres. You can see the rocksdb keys which were used in this file (this key format can be used to fetch blocks based on height): https://github.com/LiskHQ/lisk-sdk/blob/c9c94a4fc0faaf72193d9689df946b6a87511887/elements/lisk-chain/src/data_access/storage.ts#L380

sacOO7 commented 3 years ago

Related blog https://lisk.com/blog/development/benchmarking-lisk-core-v3.0.0-against-lisk-core-v2.1.6-0 , it says rocksdb provides better query than postgres

sacOO7 commented 3 years ago

I was thinking about filering only cenrtain type of transations/accounts/multiSigAccounts based on data we sign using ldex. I am not sure if we really need all types of accounts/transactions/multiSigAccounts to be imported, any thoughts?