zerodevapp / aa-benchmark

40 stars 15 forks source link

Account for L2 Gas in AA Benchmarks #6

Open ankurdubey521 opened 11 months ago

ankurdubey521 commented 11 months ago

Given that a lot of AA activity happens on rollups such as Optimism and Arbtirum, the compute gas cost might not be indicative of the actual fee paid for transaction execution since L1 gas is responsible for the majority of costs of transaction execution on these chains.

I propose that the benchmarks be updated to reflect the call data / L1 costs as well.

leekt commented 11 months ago

thanks for pointing this out, and yes i 100% agree on this point and also added a calldata size calculation a while ago but didn't had time to look into how call data costs are actually calculated, do you have any good idea how to deal with it?

my concern here was that L2 price usually depends on 1) calldata bytes(length and amount of 0s in bytes) 2) L1/L2 gas price

1 is calculated easily, as you pointed out we have calldata size but 2 is something dynamic that i am not sure how we should deal with it.

any idea?

leekt commented 11 months ago

Also, the fact that pvg calculation defers per bundler is one another issue 🤦

ankurdubey521 commented 11 months ago

Maybe it could make sense to start by showing the calldata length and some indicator of 0 byte distribution for each smart account?

Given that this repository aims to provide a comparison between different smart account implementations, I would assume L1 has fluctuations would affect each implementation proportionally to the calldata cost (post compression).

For the sake of comparison we could maybe also assume that all ops are submitted by the same bundler.

leekt commented 11 months ago

good. let's start with that I'll make a pr later this week :) or do you want to make a pr?

ankurdubey521 commented 11 months ago

Sure! Can take a stab at it. Please assign the issue to me

leekt commented 11 months ago

done

wilsoncusack commented 11 months ago

I suspect this will pretty much just come down what all gets packed in the signature field?

ankurdubey521 commented 11 months ago

Yup

ankurdubey521 commented 11 months ago

Also apologies for the rasio silence here, I should have something here this week