search

crypto-org-chain / cronos

Cronos is the first Ethereum-compatible blockchain network built on Cosmos SDK technology. Cronos aims to massively scale the DeFi, GameFi, and overall Web3 user community by providing builders with the ability to instantly port apps and crypto assets from other chains while benefiting from low transaction fees, high throughput, and fast finality.
Other
295 stars 237 forks source link

Problem: test transactions can't be saved and reused #1575

Closed mmsqe closed 2 months ago

mmsqe commented 2 months ago

Solution:

👮🏻👮🏻👮🏻 !!!! REFERENCE THE PROBLEM YOUR ARE SOLVING IN THE PR TITLE AND DESCRIBE YOUR SOLUTION HERE !!!! DO NOT FORGET !!!! 👮🏻👮🏻👮🏻

Closes: #1573

nix run .#stateless-testcase generic-gen '{
  "outdir": "/tmp/data",
  "validators": 4,
  "fullnodes": 4,
  "validator_generate_load": true,
  "num_accounts": 8000,
  "num_txs": 1,
  "app_patch": {
      "mempool.max-txs": -1,
      "evm.block-stm-pre-estimate": false
  },
  "config_patch": {
      "mempool.size": 60000
  }
}'

and 

stateless-testcase run --num-workers 500 --sleep 0
block 4 701 2024-09-17T01:20:49.614146132Z 534.1916978532505
block 5 7761 2024-09-17T01:20:51.64919105Z 3813.6748818822184
block 6 7761 2024-09-17T01:21:28.398110803Z 211.18988561268972
block 7 7761 2024-09-17T01:21:31.54134243Z 2469.1145928776496
block 8 7761 2024-09-17T01:21:34.476833542Z 2643.850722076818
block 9 7761 2024-09-17T01:21:37.574382043Z 2505.529371771036
block 10 7176 2024-09-17T01:21:40.666338961Z 2320.8609695610157

diable validator_generate_load

nix run .#stateless-testcase generic-gen '{
  "outdir": "/tmp/data",
  "validators": 3,
  "fullnodes": 7,
  "validator_generate_load": false,
  "num_accounts": 12000,
  "num_txs": 1,
  "app_patch": {
      "mempool.max-txs": -1,
      "evm.block-stm-pre-estimate": false
  },
  "config_patch": {
      "mempool.size": 60000
  }
}'

to run

stateless-testcase run --num-workers 10000 --sleep 0
[3208.6092099197745, 3020.13226941868, 2939.3986219935646]
block 11 7761 2024-09-17T04:24:24.579311546Z 2760.620974316728
block 12 7761 2024-09-17T04:24:27.149066255Z 3020.13226941868
block 13 7761 2024-09-17T04:24:29.789402548Z 2939.3986219935646
block 14 7761 2024-09-17T04:24:32.208207883Z 3208.6092099197745
block 15 7761 2024-09-17T04:24:34.869839009Z 2915.880181783207

PR Checklist:

Thank you for your code, it's appreciated! :)

Summary by CodeRabbit

Summary by CodeRabbit

coderabbitai[bot] commented 2 months ago

[!WARNING]

Rate limit exceeded

@yihuang has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 5 minutes and 38 seconds before requesting another review.

How to resolve this issue? After the wait time has elapsed, a review can be triggered using the `@coderabbitai review` command as a PR comment. Alternatively, push new commits to this PR. We recommend that you space out your commits to avoid hitting the rate limit.
How do rate limits work? CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our [FAQ](https://coderabbit.ai/docs/faq) for further information.
Commits Files that changed from the base of the PR and between 5ea96591d32e7c30ee685c1dc7a889134e61de0f and d99d6fe1c51e0e802539ea017217f419e744ec8b.

Walkthrough

The changes enhance transaction handling and node initialization within the sendtx.py and stateless.py files. The sendtx.py file introduces asynchronous transaction signing and writing, while stateless.py adds new parameters and CLI commands for generating transactions. Additionally, the transition to a stateless-testcase framework is reflected in the flake.nix and overlay.nix files, indicating a structural rebranding.

Changes

File Path Change Summary
testground/benchmark/benchmark/stateless.py Enhanced transaction management with new parameters and functions for generating transactions. Introduced CLI commands for transaction generation. Modified do_run to load transactions from an output directory.
testground/benchmark/benchmark/sendtx.py Improved transaction handling with asynchronous signing and writing. Refactored prepare_txs for concurrent processing and enhanced error handling.
testground/benchmark/flake.nix Changed default package from pkgs.testground-testcase to pkgs.stateless-testcase, updating configurations accordingly.
testground/benchmark/overlay.nix Renamed exported entities from testground-testcase to stateless-testcase, indicating a shift in architecture.
nix/testground-image.nix Updated Docker image configuration to replace testground-testcase with stateless-testcase.

Assessment against linked issues

Objective Addressed Explanation
Generate all test accounts and transactions in the gen phase. (#1573) ✅

Possibly related PRs

Poem

🐰 In the realm of code, where transactions play,
New features emerge, brightening the day.
With each hop and tweak, our benchmarks refine,
A stateless adventure, where all will align.
So gather your data, let the metrics unfold,
In this world of testing, new stories are told! 🌟

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share - [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)
Tips ### Chat There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai): - Review comments: Directly reply to a review comment made by CodeRabbit. Example: -- `I pushed a fix in commit , please review it.` -- `Generate unit testing code for this file.` - `Open a follow-up GitHub issue for this discussion.` - Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples: -- `@coderabbitai generate unit testing code for this file.` -- `@coderabbitai modularize this function.` - PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples: -- `@coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.` -- `@coderabbitai read src/utils.ts and generate unit testing code.` -- `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.` -- `@coderabbitai help me debug CodeRabbit configuration file.` Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. ### CodeRabbit Commands (Invoked using PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai full review` to do a full review from scratch and review all the files again. - `@coderabbitai summary` to regenerate the summary of the PR. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository. - `@coderabbitai help` to get help. ### Other keywords and placeholders - Add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. - Add `@coderabbitai summary` to generate the high-level summary at a specific location in the PR description. - Add `@coderabbitai` anywhere in the PR title to generate the title automatically. ### CodeRabbit Configuration File (`.coderabbit.yaml`) - You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository. - Please see the [configuration documentation](https://docs.coderabbit.ai/guides/configure-coderabbit) for more information. - If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json` ### Documentation and Community - Visit our [Documentation](https://coderabbit.ai/docs) for detailed information on how to use CodeRabbit. - Join our [Discord Community](https://discord.com/invite/GsXnASn26c) to get help, request features, and share feedback. - Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.
codecov[bot] commented 2 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 15.24%. Comparing base (3c8be96) to head (d99d6fe). Report is 1 commits behind head on main.

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/crypto-org-chain/cronos/pull/1575/graphs/tree.svg?width=650&height=150&src=pr&token=92w0Dz1Coj&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=crypto-org-chain)](https://app.codecov.io/gh/crypto-org-chain/cronos/pull/1575?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=crypto-org-chain) ```diff @@ Coverage Diff @@ ## main #1575 +/- ## ======================================= Coverage 15.24% 15.24% ======================================= Files 67 67 Lines 4874 4874 ======================================= Hits 743 743 Misses 4037 4037 Partials 94 94 ```
yihuang commented 2 months ago 
diff --git a/testground/benchmark/benchmark/stateless.py b/testground/benchmark/benchmark/stateless.py
index 5e4db258..4fb54d0e 100644
--- a/testground/benchmark/benchmark/stateless.py
+++ b/testground/benchmark/benchmark/stateless.py
@@ -6,6 +6,7 @@ import socket
 import subprocess
 import tarfile
 import tempfile
+import threading
 import time
 from pathlib import Path
 from typing import List
@@ -220,12 +221,26 @@ def _gen_txs(
     num_txs: int = 1000,
 ):
     outdir = Path(outdir)
-    for global_seq in range(nodes):
+
+    # create dir in advance to avoid conflict in parallel task
+    (outdir / transaction.TXS_DIR).mkdir(parents=True, exist_ok=True)
+
+    def job(global_seq: int):
         print("generating", num_accounts * num_txs, "txs for node", global_seq)
         txs = transaction.gen(global_seq, num_accounts, num_txs)
         transaction.save(txs, outdir, global_seq)
         print("saved", len(txs), "txs for node", global_seq)

+    threads = [
+        threading.Thread(target=job, args=(global_seq,)) for global_seq in range(nodes)
+    ]
+
+    for t in threads:
+        t.start()
+
+    for t in threads:
+        t.join()
+

 def do_run(
     datadir: Path, home: Path, cronosd: str, group: str, global_seq: int, cfg: dict

I actually tested this parallel implementation, CPU usage never pass 100%, and there's zero speedup, even several seconds slower, the reason is eth-account rely on pure python cryptographic implementation, so it's all protected by GIL. @mmsqe

mmsqe commented 2 months ago

yes, I thought need generate tx_{acci}{tx_i}.json file in parallel avoid regenerate when changing num_accts and num_txs

yihuang commented 2 months ago

yes, I thought need generate tx_{acci}{tx_i}.json file one by one avoid regenerate when changing different configs

I think we just backup the whole txs directory to reuse it. When num_accounts changed, just generate new ones, it's not that slow to generate actually.