search

celo-org / celo-blockchain

Official repository for the golang Celo Blockchain
https://celo.org
GNU Lesser General Public License v3.0
560 stars 198 forks source link

RSDB: metrics and performance improvements #2184

Closed carterqw2 closed 1 year ago

carterqw2 commented 1 year ago

Description

Reuses the existing LevelDB implementation from the ethdb/leveldb module to collect LevelDB metrics for Roundstate DB as well and introduces additional metrics related to RLP-encoding.

Other changes

Tested

Tested on the testnet.

Backwards compatibility

No breaking changes.

github-actions[bot] commented 1 year ago

Coverage from tests in ./e2e_test/... for ./consensus/istanbul/... at commit 0ed594e4363462542e8e879afd539b93e3bc1772

coverage: 48.6% of statements across all listed packages 
coverage:  59.8% of statements in consensus/istanbul
coverage:  42.7% of statements in consensus/istanbul/announce
coverage:  55.7% of statements in consensus/istanbul/backend
coverage:   0.0% of statements in consensus/istanbul/backend/backendtest
coverage:  24.3% of statements in consensus/istanbul/backend/internal/replica
coverage:  58.1% of statements in consensus/istanbul/core
coverage:  50.0% of statements in consensus/istanbul/db
coverage:   0.0% of statements in consensus/istanbul/proxy
coverage:  64.4% of statements in consensus/istanbul/uptime
coverage:  51.8% of statements in consensus/istanbul/validator
coverage:  79.2% of statements in consensus/istanbul/validator/random
github-actions[bot] commented 1 year ago

5861 passed, 1 failed, 45 skipped

Test failures:
  TestPriorityClient: geth
    les_test.go:121: Initializing geth: [--networkid=42 init ./testdata/clique.json]
test_cmd.go:262: (stderr:31) INFO [09-11|10:54:27.170] Maximum peer count                       ETH=175 LES=0 total=175
test_cmd.go:262: (stderr:31) INFO [09-11|10:54:27.171] Set global gas inflation rate            rate=1.300
test_cmd.go:262: (stderr:31) INFO [09-11|10:54:27.171] Set global gas cap                       cap=25,000,000
test_cmd.go:262: (stderr:31) INFO [09-11|10:54:27.171] Allocated cache and file handles         database=/tmp/geth-test3910766448/celo/chaindata cache=16.00MiB handles=16
test_cmd.go:262: (stderr:31) INFO [09-11|10:54:27.188] Writing custom genesis block
test_cmd.go:262: (stderr:31) INFO [09-11|10:54:27.189] Persisted trie from memory database      nodes=4 size=566.00B time="59.086µs" gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=-82.00B
test_cmd.go:262: (stderr:31) INFO [09-11|10:54:27.189] Successfully wrote genesis state         database=chaindata                               hash=9195f2..122dcd
test_cmd.go:262: (stderr:31) INFO [09-11|10:54:27.189] Allocated cache and file handles         database=/tmp/geth-test3910766448/celo/lightchaindata cache=16.00MiB handles=16
test_cmd.go:262: (stderr:31) INFO [09-11|10:54:27.202] Writing custom genesis block
test_cmd.go:262: (stderr:31) INFO [09-11|10:54:27.203] Persisted trie from memory database      nodes=4 size=566.00B time="31.055µs" gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=-82.00B
test_cmd.go:262: (stderr:31) INFO [09-11|10:54:27.203] Successfully wrote genesis state         database=lightchaindata                          hash=9195f2..122dcd
test_cmd.go:262: (stderr:31) INFO [09-11|10:54:27.203] Allocated cache and file handles         database=/tmp/geth-test3910766448/celo/lightestchaindata cache=16.00MiB handles=16
test_cmd.go:262: (stderr:31) INFO [09-11|10:54:27.217] Writing custom genesis block
test_cmd.go:262: (stderr:31) INFO [09-11|10:54:27.217] Persisted trie from memory database      nodes=4 size=566.00B time="21.065µs" gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=-82.00B
test_cmd.go:262: (stderr:31) INFO [09-11|10:54:27.218] Successfully wrote genesis state         database=lightestchaindata                       hash=9195f2..122dcd
les_test.go:130: Importing keys to geth
test_cmd.go:262: (stderr:32) INFO [09-11|10:54:27.252] Maximum peer count                       ETH=175 LES=0 total=175
test_cmd.go:262: (stderr:32) INFO [09-11|10:54:27.253] Set global gas inflation rate            rate=1.300
test_cmd.go:262: (stderr:32) INFO [09-11|10:54:27.253] Set global gas cap                       cap=25,000,000
les_test.go:99: Starting lightserver with rpc: [--networkid=42 --port=0 --ipcpath geth-1.ipc --allow-insecure-unlock --datadir /tmp/geth-test3910766448 --password ./testdata/password.txt --unlock 0x02f0d131f1f97aef08aec6e3291b957d9efe7105 --mine --miner.validator 0x02f0d131f1f97aef08aec6e3291b957d9efe7105 --tx-fee-recipient 0x02f0d131f1f97aef08aec6e3291b957d9efe7105 --light.serve=100 --light.maxpeers=1 --nodiscover --nat=extip:127.0.0.1 --verbosity=4]
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.505] Maximum peer count                       ETH=175 LES=1 total=176
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.506] FS scan times                            list="39.374µs" set="4.524µs" diff="1.527µs"
test_cmd.go:262: (stderr:33) WARN [09-11|10:54:27.506] LES server cannot serve old transaction status and cannot connect below les/4 protocol version if transaction lookup index is limited
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.506] Sanitizing Go's GC trigger               percent=100
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.506] Set global gas inflation rate            rate=1.300
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.506] Set global gas cap                       cap=25,000,000
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.506] Allocated trie memory caches             clean=154.00MiB dirty=256.00MiB
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.506] Allocated cache and file handles         database=/tmp/geth-test3910766448/celo/chaindata cache=512.00MiB handles=524,288
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.552] Chain freezer table opened               database=/tmp/geth-test3910766448/celo/chaindata/ancient table=receipts items=0 size=0.00B
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.555] Chain freezer table opened               database=/tmp/geth-test3910766448/celo/chaindata/ancient table=diffs    items=0 size=0.00B
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.557] Chain freezer table opened               database=/tmp/geth-test3910766448/celo/chaindata/ancient table=headers  items=0 size=0.00B
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.561] Chain freezer table opened               database=/tmp/geth-test3910766448/celo/chaindata/ancient table=hashes   items=0 size=0.00B
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.563] Chain freezer table opened               database=/tmp/geth-test3910766448/celo/chaindata/ancient table=bodies   items=0 size=0.00B
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.563] Opened ancient database                  database=/tmp/geth-test3910766448/celo/chaindata/ancient readonly=false
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.563] Current full block not old enough        number=0 hash=9195f2..122dcd delay=90000
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.564] Initialised chain configuration          config="{ChainID: 15 Homestead: 0 DAO:  DAOSupport: false EIP150: 0 EIP155: 0 EIP158: 0 Byzantium: 0 Constantinople: 0 Petersburg: 0 Istanbul:  Churrito: , Donut: , Espresso: , Gingerbread: , Gingerbread P2: , Engine: istanbul}"
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.564] Setting up Istanbul consensus engine
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.604] Initialising Ethereum protocol           versions=[67] network=42 dbversion=
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.605] Loading Last State: 0
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.606] Loaded most recent local header          number=0 hash=9195f2..122dcd td=1 age=54y5mo3w
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.606] Loaded most recent local full block      number=0 hash=9195f2..122dcd td=1 age=54y5mo3w
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.606] Loaded most recent local fast block      number=0 hash=9195f2..122dcd td=1 age=54y5mo3w
test_cmd.go:262: (stderr:33) WARN [09-11|10:54:27.606] Failed to load snapshot, regenerating    err="missing or corrupted snapshot"
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.606] Rebuilding state snapshot
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.606] Journalled generator progress            progress=empty
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.606] Start snapshot generation                root=8a7a77..f86ff6
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.606] Resuming state snapshot generation       root=8a7a77..f86ff6 accounts=0 slots=0 storage=0.00B elapsed="193.668µs"
test_cmd.go:262: (stderr:33) ERROR[09-11|10:54:27.606] Error invoking evm function: EVM call failure to=0x000000000000000000000000000000000000ce10 method=getAddressFor input=0xdd9272338b91d837e1684f7353d73b6197230894243cf869282f722841df96b441303f37 maxgas=100,000 err="execution reverted"            message="No Implementation set"
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.606] Registry contract not yet deployed       function=blockGasLimit registryId=0x8b91d837e1684f7353d73b6197230894243cf869282f722841df96b441303f37
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.606] Error calling blockGasLimit              err="Registry not deployed"         contract=0x8b91d837e1684f7353d73b6197230894243cf869282f722841df96b441303f37
test_cmd.go:262: (stderr:33) ERROR[09-11|10:54:27.606] Error invoking evm function: EVM call failure to=0x000000000000000000000000000000000000ce10 method=getAddressFor input=0xdd92723348766a30bcca8b3c8b03b4e4b1f6510aee73f7b41aa6c7578a97040d20334f42 maxgas=100,000 err="execution reverted"            message="No Implementation set"
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.606] Registry contract not yet deployed       function=gasPriceMinimumFloor registryId=0x48766a30bcca8b3c8b03b4e4b1f6510aee73f7b41aa6c7578a97040d20334f42
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.606] Regenerated state range                  kind=storage prefix=0x6fc8292a2675fd2df398d2f757c44afe1b96e4c7ba26dac7ed67cdef77799ef404 root=09f508..8cb885 last=0x count=1 created=1 updated=0 untouched=0 deleted=0
test_cmd.go:262: (stderr:33) ERROR[09-11|10:54:27.606] Error invoking evm function: EVM call failure to=0x000000000000000000000000000000000000ce10 method=getAddressFor input=0xdd9272338b91d837e1684f7353d73b6197230894243cf869282f722841df96b441303f37 maxgas=100,000 err="execution reverted"            message="No Implementation set"
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.606] Regenerated state range                  kind=account prefix=0x61 root=8a7a77..f86ff6 last=0x count=2 created=2 updated=0 untouched=0 deleted=0
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.606] Journalled generator progress            progress=done
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.606] Registry contract not yet deployed       function=intrinsicGasForAlternativeFeeCurrency registryId=0x8b91d837e1684f7353d73b6197230894243cf869282f722841df96b441303f37
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.606] Generated state snapshot                 accounts=2 slots=1 storage=230.00B elapsed="531.488µs"
test_cmd.go:262: (stderr:33) ERROR[09-11|10:54:27.606] Error invoking evm function: EVM call failure to=0x000000000000000000000000000000000000ce10 method=getAddressFor input=0xdd9272338f897d9543fdc9082789208180c3e3d11f7473aaef3e249ab0020ea0c761c1b6 maxgas=100,000 err="execution reverted"            message="No Implementation set"
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.606] Registry contract not yet deployed       function=getWhitelist                          registryId=0x8f897d9543fdc9082789208180c3e3d11f7473aaef3e249ab0020ea0c761c1b6
test_cmd.go:262: (stderr:33) ERROR[09-11|10:54:27.606] getWhitelist invocation failed           err="Registry not deployed"
test_cmd.go:262: (stderr:33) ERROR[09-11|10:54:27.606] Error invoking evm function: EVM call failure to=0x000000000000000000000000000000000000ce10 method=getAddressFor input=0xdd927233d7e89ade8430819f08bf97a087285824af3351ee12d72a2d132b0c6c0687bfaf maxgas=100,000 err="execution reverted"            message="No Implementation set"
test_cmd.go:262: (stderr:33) DEBUG[09-11|10:54:27.606] Reinjecting stale transactions           count=0
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.606] Regenerated local transaction journal    transactions=0 accounts=0
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.615] Allocated fast sync bloom                size=512.00MiB
test_cmd.go:262: (stderr:33) WARN [09-11|10:54:27.616] Error reading unclean shutdown markers   error="leveldb: not found"
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.617] Allocated cache and file handles         database=/tmp/geth-test3910766448/celo/les.server cache=16.00MiB  handles=16
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:27.620] Initialized state bloom                  items=5 errorrate=0.000 elapsed=4.929ms
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:28.828] Checkpoint oracle is not enabled
test_cmd.go:262: (stderr:33) INFO [09-11|10:54:28.828] Starting peer-to-peer node               instance=celo/v1.8.0-unstable/linux-amd64/go1.18.10
les_test.go:115: lightserver rpc connect to /tmp/geth-test3910766448/geth-1.ipc: dial unix /tmp/geth-test3910766448/geth-1.ipc: connect: no such file or directory
This test report was produced by the test-summary action.  Made with ❤️ in Cambridge.
codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 59.67% and project coverage change: +0.12% :tada:

Comparison is base (cd2e13a) 55.05% compared to head (25256e6) 55.18%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #2184 +/- ## ========================================== + Coverage 55.05% 55.18% +0.12% ========================================== Files 679 680 +1 Lines 114336 114370 +34 ========================================== + Hits 62950 63116 +166 + Misses 47495 47378 -117 + Partials 3891 3876 -15 ``` | [Files Changed](https://app.codecov.io/gh/celo-org/celo-blockchain/pull/2184?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=celo-org) | Coverage Δ | | |---|---|---| | [ethdb/leveldb/celo\_leveldb.go](https://app.codecov.io/gh/celo-org/celo-blockchain/pull/2184?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=celo-org#diff-ZXRoZGIvbGV2ZWxkYi9jZWxvX2xldmVsZGIuZ28=) | `0.00% <0.00%> (ø)` | | | [ethdb/leveldb/leveldb.go](https://app.codecov.io/gh/celo-org/celo-blockchain/pull/2184?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=celo-org#diff-ZXRoZGIvbGV2ZWxkYi9sZXZlbGRiLmdv) | `14.62% <0.00%> (-0.05%)` | :arrow_down: | | [consensus/istanbul/core/roundstate\_db.go](https://app.codecov.io/gh/celo-org/celo-blockchain/pull/2184?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=celo-org#diff-Y29uc2Vuc3VzL2lzdGFuYnVsL2NvcmUvcm91bmRzdGF0ZV9kYi5nbw==) | `61.63% <72.54%> (+2.99%)` | :arrow_up: | ... and [25 files with indirect coverage changes](https://app.codecov.io/gh/celo-org/celo-blockchain/pull/2184/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=celo-org)

:umbrella: View full report in Codecov by Sentry.

:loudspeaker: Have feedback on the report? Share it here.