[Testnet] Benchmarking zksync testnet and optimize performances

thomas-nguy commented 12 months ago

Benchmark the zk sync testnet network
Understand which parameters in config can be modified to tune l2 performances

crypto-matto commented 12 months ago

Current Findings

Benchmark works perfectly fine for 30TPS.

Type	Benchmark TPS	Average TPS (Network)	Average TPS (Client)	Success Rate	Gas Used / s	Total Gas Used	Average Response Time(s)	Longest Response Time(s)
Native Token Transfer	30	30.303	28.571	2000/2000: 100.00%	5174138	341493128	7.119	7.171
ERC20 Transfer	30	30.303	28.571	2000/2000: 100.00%	5273752	348067632	7.124	7.189
Mint	30	28.157	27.000	1971/2000: 98.55%	6342941	444005909	7.963	14.289
Swap	30	29.851	29.412	2000/2000: 100.00%	8637026	578680798	7.119	7.261

At 40 TPS, I start to see a drop of success rate. But the transactions are not dropped. It's because the benchmark tool has a timeout time for 2 mins, and there are occasions where new blocks are generated after several mins. Under higher TPS, the transactions will be kept in the mempool and not proceeded, until a new block is generated.

e.g. There's a time diff of 158s between block 3228 & 3229

Block 3226 | Timestamp: 1689051142 | tx: 65
Block 3227 | Timestamp: 1689051143 | tx: 24
Block 3228 | Timestamp: 1689051144 | tx: 0
Block 3229 | Timestamp: 1689051302 | tx: 208
Block 3230 | Timestamp: 1689051303 | tx: 42
Block 3231 | Timestamp: 1689051303 | tx: 0
Block 3232 | Timestamp: 1689051316 | tx: 74

Possible Reasons

Miniblock will get sent to L1 if:

deadline has expired
miniblock is full Will investigate

crypto-matto commented 11 months ago

Updates on delayed block generation under high TPS pressure

Due to the fact that the situation cannot be replicated on the local environment, and no suspicious logs are found, the below potential assumptions to this are dropped:

deadline has expired
miniblock is full

Instead, after the server upgraded & stablised on testnet, the above situation is gone. We instead suspect it was due to:

An instability of the zkSync Server
Block generation will suspend from time to time during the zkSync server is down & rebooted

Figures

As zkSync claims the sequencer can support up to 100 TPS, we benchmark the data until 100 TPS.	Type	Benchmark TPS	Average TPS (Network)	Average TPS (Client)	Success Rate	Gas Used / s	Total Gas Used	Average Response Time(s)
Native Token Transfer	100	90.909	71.429	2000/2000: 100.00%	15510680	341234972	7.209	8.535
ERC20 Transfer	100	64.516	54.054	2000/2000: 100.00%	11220081	347822528	7.329	9.308
Mint	100	48.780	45.455	2000/2000: 100.00%	10987765	450498376	8.766	21.648
Swap	100	60.606	52.632	2000/2000: 100.00%	17574506	562384214	7.354	14.268

Conclusion

The zkSync performance is bumped significantly, and for simple transfer transactions, it is also justify to claim that it can support up to 100 TPS

crypto-matto commented 11 months ago

Detailed Figures on various tx types

Normal Transfer

Benchmark TPS	Average TPS (Network)	Average TPS (Client)	Success Rate	Gas Used / s	Total Gas Used	Average Response Time(s)	Longest Response Time(s)
50	51.282	46.512	2000/2000: 100.00%	8750350	341263656	7.295	9.036
60	58.824	51.282	2000/2000: 100.00%	10037166	341263656	7.331	9.884
70	71.429	60.606	2000/2000: 100.00%	12187977	341263368	7.384	9.548
80	74.074	58.824	2000/2000: 100.00%	12639400	341263800	7.342	10.125
90	80.000	66.667	2000/2000: 100.00%	13649421	341235548	7.253	8.790
100	90.909	71.429	2000/2000: 100.00%	15510680	341234972	7.209	8.535

ERC20 Transfer

Benchmark TPS	Average TPS (Network)	Average TPS (Client)	Success Rate	Gas Used / s	Total Gas Used	Average Response Time(s)	Longest Response Time(s)
50	50.000	44.444	2000/2000: 100.00%	8696295	347851816	7.333	11.969
60	62.290	52.189	2000/2000: 100.00%	10832338	335802504	7.415	9.722
70	68.966	66.667	2000/2000: 100.00%	11992800	347791224	7.250	8.434
80	66.667	55.556	2000/2000: 100.00%	11593064	347791944	7.198	7.723
90	76.923	62.500	2000/2000: 100.00%	13376574	347790936	7.231	9.615
100	64.516	54.054	2000/2000: 100.00%	11220081	347822528	7.329	9.308

Mint

Benchmark TPS	Average TPS (Network)	Average TPS (Client)	Success Rate	Gas Used / s	Total Gas Used	Average Response Time(s)	Longest Response Time(s)
50	41.667	40.000	2000/2000: 100.00%	9385382	450498376	7.991	15.677
60	46.512	44.444	2000/2000: 100.00%	10476719	450498952	8.008	16.415
70	48.780	45.455	2000/2000: 100.00%	10987786	450499240	9.261	23.914
80	51.282	46.512	2000/2000: 100.00%	11551262	450499240	7.908	14.524
90	46.512	43.478	2000/2000: 100.00%	10476706	450498376	9.866	28.374
100	48.780	45.455	2000/2000: 100.00%	10987765	450498376	8.766	21.648

Swap

Benchmark TPS	Average TPS (Network)	Average TPS (Client)	Success Rate	Gas Used / s	Total Gas Used	Average Response Time(s)	Longest Response Time(s)
50	48.780	46.512	2000/2000: 100.00%	14100318	578113038	7.283	9.285
60	54.054	51.282	2000/2000: 100.00%	15624700	578113902	7.458	9.787
70	58.824	54.054	2000/2000: 100.00%	17001288	578043796	7.539	15.119
80	60.606	55.556	2000/2000: 100.00%	17514323	577972682	7.245	7.942
90	54.054	51.282	2000/2000: 100.00%	15622805	578043796	8.825	21.290
100	60.606	52.632	2000/2000: 100.00%	17574506	562384214	7.354	14.268

crypto-matto commented 11 months ago

RAM usage over time

The server generates witnesses for blocks right after the benchmark starts, and the RAM goes up constantly. It reaches a stable % after certain point, regardless of any new blast of benchmarking, or in idle.

cronos-labs / cronos-zkevm