lbryio / lbcd

An alternative full node implementation of LBRY's blockchain written in Go (golang)
https://lbry.com/
ISC License
38 stars 25 forks source link

Claimtrie progress logging. #57

Closed moodyjon closed 2 years ago

moodyjon commented 2 years ago

Fixes: https://github.com/lbryio/lbcd/issues/50

I copied claimtrie/logger.go from netsync/blocklogger.go. Copyright comment from there left intact. Let me know if there is a better way to handle derivative works like this.

I did not find a way to handle percentage progress... The pebble stats API stuff seems to be private, so I did not try to use that to estimate the number of records.

Example output:

2022-06-14 13:32:48.121 [INF] MAIN: Version 0.0.0-local.0+67ce441247ecc148a9f21ef1798bb11269103c72-dirty
2022-06-14 13:32:48.121 [INF] MAIN: Loading block database from '/Users/swdev1/Library/Application Support/Lbcd/data/mainnet/blocks_ffldb'
2022-06-14 13:32:48.470 [INF] MAIN: Block database loaded
2022-06-14 13:32:51.472 [INF] SRVR: Can't discover UPnP-enabled device: write udp4 0.0.0.0:54154->239.255.255.250:1900: i/o timeout
2022-06-14 13:32:51.479 [INF] INDX: Transaction index is enabled
2022-06-14 13:32:51.480 [INF] INDX: Committed filter index is enabled
2022-06-14 13:32:52.000 [INF] LBRY: Normalizing strings via Go. Casefold and NFD table versions: 11.0.0 (from ICU 63.2)
2022-06-14 13:32:52.001 [INF] LBRY: Building the entire claim trie in RAM...
2022-06-14 13:32:56.030 [INF] LBRY: Spending claim but missing existing claim with TXO bd183d94dc0b2c38d500838f4ab7c93b643123f46e87214d3e37f5e9e0d4baf5:0, Name: 1791l-trumpcnn, ID: 6a28cbbe64c3c54113770705c928e80a138ddae9
2022-06-14 13:33:02.000 [INF] LBRY: Processed 1915480 claim names in the last 10s (total 1915480)
2022-06-14 13:33:12.001 [INF] LBRY: Processed 2113173 claim names in the last 10s (total 4028653)
2022-06-14 13:33:22.001 [INF] LBRY: Processed 1787925 claim names in the last 10s (total 5816578)
2022-06-14 13:33:28.681 [INF] MAIN: RAM: using 1.3 GB with 3.0 available, DISK: using 123.0 GB with 1164.9 available
2022-06-14 13:33:32.001 [INF] LBRY: Processed 1596516 claim names in the last 10s (total 7413094)
2022-06-14 13:33:42.001 [INF] LBRY: Processed 1519922 claim names in the last 10s (total 8933016)
2022-06-14 13:33:52.004 [INF] LBRY: Processed 1667066 claim names in the last 10s (total 10600082)
2022-06-14 13:33:52.753 [INF] LBRY: Spending claim but missing existing claim with TXO 546a1b99218f5935d6d11d6a5632ba199b24d6dbb6c07e75bea1089eee23987d:0, Name: ggminingpool-xzc, ID: 0b9e8ea179d6f68d3b37ab0e636af6ab1b93b52e
2022-06-14 13:33:52.753 [INF] LBRY: Spending claim but missing existing claim with TXO 5dff52209029a63547f8805c3e92fce87e3744a15f6539a12011fe0c421ca132:0, Name: ggminingpoolxzc, ID: 3fa8d7113bd4873e79ec02019bea3c7840f0deee
2022-06-14 13:34:02.004 [INF] LBRY: Processed 1486449 claim names in the last 10s (total 12086531)
2022-06-14 13:34:08.598 [INF] MAIN: RAM: using 2.3 GB with 2.7 available, DISK: using 123.0 GB with 1164.9 available
2022-06-14 13:34:12.004 [INF] LBRY: Processed 1337525 claim names in the last 10s (total 13424056)
2022-06-14 13:34:22.005 [INF] LBRY: Processed 1376697 claim names in the last 10s (total 14800753)
2022-06-14 13:34:29.657 [INF] LBRY: Spending claim but missing existing claim with TXO 022a25b42e0327d6bf7a741cd3bae34404453549635d916e5b05eb15edb05019:1, Name: redbullfourthphase, ID: f054e085e8d38c0a0e63a472b033e42d437a4014
2022-06-14 13:34:32.005 [INF] LBRY: Processed 1455552 claim names in the last 10s (total 16256305)
2022-06-14 13:34:42.005 [INF] LBRY: Processed 1426823 claim names in the last 10s (total 17683128)
2022-06-14 13:34:45.046 [INF] LBRY: Spending claim but missing existing claim with TXO d549351f731019573d921f5db8e2844449edf3011073b3e5f4ce9b0d9aee5490:1, Name: timcast-sweden-day1, ID: 247895d4b9389b74038ea5a4fd807cafc97ede53
2022-06-14 13:34:48.822 [INF] MAIN: RAM: using 2.8 GB with 2.7 available, DISK: using 123.0 GB with 1164.9 available
2022-06-14 13:34:48.850 [INF] LBRY: Spending claim but missing existing claim with TXO 36a719a156a1df178531f3c712b8b37f8e7cc3b36eea532df961229d936272a1:0, Name: two, ID: bb6168c9a56286999c7c938c7b8c5afb221d38d0
2022-06-14 13:34:52.005 [INF] LBRY: Processed 1134572 claim names in the last 10s (total 18817700)
2022-06-14 13:35:02.005 [INF] LBRY: Processed 1028133 claim names in the last 10s (total 19845833)
2022-06-14 13:35:23.789 [INF] CHAN: Loading block index...
2022-06-14 13:35:28.856 [INF] MAIN: RAM: using 2.8 GB with 2.7 available, DISK: using 123.0 GB with 1162.8 available
2022-06-14 13:35:38.269 [INF] CHAN: Chain state (height 1176375, hash 47098eddcdd048c47830428603983b73646c3dcd019c33ba0595a63aa84d45fc, totaltx 80627449, work 44778323276798910409405)
coveralls commented 2 years ago

Pull Request Test Coverage Report for Build 2497077944


Changes Missing Coverage Covered Lines Changed/Added Lines %
claimtrie/node/log.go 0 6 0.0%
claimtrie/logger.go 19 35 54.29%
<!-- Total: 25 47 53.19% -->
Files with Coverage Reduction New Missed Lines %
peer/peer.go 8 75.92%
<!-- Total: 8 -->
Totals Coverage Status
Change from base Build 2450492318: -0.01%
Covered Lines: 22471
Relevant Lines: 43376

💛 - Coveralls
roylee17 commented 2 years ago

Though we don't track total number of claims at the moment, this looks like a nice improvement already.

  1. Preserving copyright comment is fine.
  2. I like how you group changes into logical commits.
  3. Another nice to have thing is prefix a category tag in the commit header. For example, this can be prefixed with "claimtrie: xxx" (I understand not every commits can be categorized cleanly though).