near / stakewars-iv

12 stars 9 forks source link

[Traffic gen]Testing keypom airdrops on Statelessnet #58

Open jaswinder6991 opened 7 months ago

jaswinder6991 commented 7 months ago

Traffic Generation Proposal

Overview

Overview of the test plan and what type of traffic the participant aims to generate. We intend to simulate airdrop conditions like mainnet. NEAR mainnet has seen a lot of traffic because airdrops. We want to creating multiple keypom airdrops. Simple account creation, FT drops, NFT drops etc.

Target traffic volume

What is the estimated traffic volume? Want to start with 10k a day and scale it up to 50k as what popular apps like Shard dog has seen on mainnet.

Don't forget to mention required tokens needed to generate the traffic for the agreed time
Not sure of this but starting with a 100 might be a good start.

Traffic should be large enough to prove/challenge scalability/stability of the network. Yes

Reproduction steps

Provide us with necessary scripts/codes (in case the engineering team needs to replay in the future) Still in progress.

staffik commented 7 months ago

Thank you for the proposal! 50k transactions a day would be less than 1 transaction per second. That does not seem large enough to challenge the network. Do you think you can generate load in the order of 200 TPS? That would resemble more the throughput that can be potentially challenging to the network.

jaswinder6991 commented 7 months ago

Thanks a lot for the feedback. I actually meant 50k claims which would be more like 100k transactions cause it is 2 step process.

I get your point though that even 100k won't be much. Yes, we can definitely do 200 TPS. Let me know next steps from your side?

walnut-the-cat commented 7 months ago

Thanks @jaswinder6991 for the prompt response. If you can generate over 200 TPS, that is good to hear! Let us take it to our side, come up with possible reward amount estimate, and get back to you to discuss the next step.

jaswinder6991 commented 7 months ago

Thanks @jaswinder6991 for the prompt response. If you can generate over 200 TPS, that is good to hear! Let us take it to our side, come up with possible reward amount estimate, and get back to you to discuss the next step.

Hey any updates here please?

walnut-the-cat commented 7 months ago

Hi @jaswinder6991 , sorry for the late response. I was sick over the last couple days. I will share the reward estimate today.

walnut-the-cat commented 7 months ago

Hello @jaswinder6991 - Here, we are sharing the reward estimate. FYI, as the team is still working on optimizing & stabilizing StatelessNet, timing of the load test will need to be coordinated with the team and is likely to be near later part of March. If you have any further questions/comment, please let us know!

Baseline reward amount

500 NEAR

Volume multiplier

1x, assuming the proposer can accomplish 200+ transactions per second.

Novelty multiplier

0.75x. The proposed load type is within the range of the traditional load types we have tested ourselves (LINK)

Outcome multiplier

1x, assuming the proposer can generated the traffic successfully at the coordinated time.

Reward estimate upon successful execution

375 NEAR (500 NEAR 1 0.75 * 1)

jaswinder6991 commented 7 months ago

Hi @walnut-the-cat, thanks for your response here. I appreciate it. I have following questions.

  1. For volume you have mentioned 1x which from the docs meaning medium, can you tell me what is the range for High volume? 500 for example? I can try hitting that.
  2. I don't think the Novelty multiplier is justified here, I agree it isn't as innovative as to get 1.5x but it is atleast 1x, here is my reasoning.
    • Creating multiple drops on keypom contract (no big deal), however claiming NFTs and FTs and other custom function call (a post on near social) has cross contract calls which I think are not in the given examples you have shared from the team.
    • I can see all simple transactions and no cross contract in the examples (maybe I am wrong, happy to know a cross contract stress test example.)
    • After thinking a little bit, another test that I can think of is calling create_account_advanced where I intend to deploy a no-std wasm contract along with account creation.

I am happy to discuss more ideas here or any advice you can provide to increase that novelty multiplier.

walnut-the-cat commented 6 months ago

Hello @jaswinder6991 , thanks for your response. Please see below for the comments.

  1. For volume you have mentioned 1x which from the docs meaning medium, can you tell me what is the range for High volume? 500 for example? I can try hitting that.

For higher volume, we are looking for magnitude of 1000. So if you can generate e.g. 1.5k tps, we can consider that with 1.5x.

  1. I don't think the Novelty multiplier is justified here, I agree it isn't as innovative as to get 1.5x but it is atleast 1x, here is my reasoning.

Please be aware that what we shared is bare minimum stress tests we are performing (not everything). Each individual action, FT transfer, social DB traffic, and NFT minting are common traffic patterns in our network and something the team has used as a baseline load test. When it comes to cross contract calls, our contract runtime team has been testing the various case with their project, e.g. local congestion control (link)

Having said that, if you have some clever plans to connect all these different patterns together to create more interesting patterns, we would love to know more about them and re-evaluate. From the initial proposal, it wasn't clear how they can be interconnected.

jaswinder6991 commented 6 months ago

Thanks @walnut-the-cat. I think that is enough for me to proceed for now. I will share my scripts here or the exact plan, we can revisit the novelty multiplier then if not, 0.75 is fine too. Thanks. I will look into high tps as well.

walnut-the-cat commented 5 months ago

Hello @jaswinder6991

We are aiming to restart currently paused StakeWars IV near end of May after on-going internal tests. (Current ETA: May 27th). Near mid-May, one of the team members will reach out to you on this issue to coordinate the schedule.

Again, I apologize for the delay. We will make sure that your time and effort are acknowledge and compensated properly.

telezhnaya commented 3 months ago

Hey @jaswinder6991, Could you please ping me on telegram? https://t.me/telezhnaya