search

klaytn / klaytn-etl

Python scripts for ETL (extract, transform and load) jobs for Klaytn blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions.
Other
21 stars 13 forks source link

Add log-percentage-step option to export_trace_group_job #6

Closed odg0318 closed 1 year ago

odg0318 commented 1 year ago

I implemented log percentage step option to export_trace_group_job. The job can be executed for a long time so we need some cases to see the progress for debugging.

This PR is helpful to debug the spent time for trace RPC API.

Symbolic Execution not available: No module named 'mythril'
pyetherchain not available: No module named 'pyetherchain'
2022-09-30 12:57:51,745 - ProgressLogger [INFO] - Started work. Items to process: 3600.
2022-09-30 12:57:52,244 - ProgressLogger [INFO] - 36 items processed. Progress is 1%.
2022-09-30 12:57:52,980 - ProgressLogger [INFO] - 72 items processed. Progress is 2%.
2022-09-30 12:57:53,564 - ProgressLogger [INFO] - 108 items processed. Progress is 3%.
2022-09-30 12:57:54,067 - ProgressLogger [INFO] - 144 items processed. Progress is 4%.
2022-09-30 12:57:54,576 - ProgressLogger [INFO] - 180 items processed. Progress is 5%.
yongchand commented 1 year ago

@odg0318 Can you set default to 10? Most users will use log percentage of 10 I think (And I believe that is standard for Ethereum-ETL too)