search

planetarium / libplanet-explorer

Libplanet Explorer now belongs to the Libplanet monorepo. This repository remains read-only as historical archive.
https://github.com/planetarium/libplanet/tree/main/Libplanet.Explorer
10 stars 20 forks source link

enrich mysqlrichstore #165

Open area363 opened 3 years ago

area363 commented 3 years ago

This PR enriches MySqlRichStore to store block and transaction data.

In this section, I’m using MySqlBulkLoader to store updated_address data in the tx.

Genesis block has around 50,000 records of updated_addresses. And so, storing these entries during preloading took around 5 minutes with the original implementation. With bulk loading, the time has decreased to about 5 seconds. However, I’ve been running into some issues bulk loading byte arrays into the table. When I just use addr.ToByteArray() here, the data is converted to a string. I’m wondering if there are other ways to bulk load byte array data. 🤔

area363 commented 3 years ago

This draft is now ready for review.

Major updates are: 1) Data is now stored in VARCHAR or BIGINT to enable bulk loading to increase insertion speed that matches RocksDB. 2) Block table is added to increase query time for ListBlocks().

Feedback needed: 1) MySql query for ListBlocks() is quite redundant but nullable limit and miner parameters make it pretty tricky to write clean, efficient code. I'd appreciate anyone's feedback on this.

dahlia commented 3 years ago

As Libplanet Explorer became placed in the Libplanet monorepo (see also: planetarium/libplanet#1244), this pull request should be sent to the Libplanet monorepo instead.