Evaluate an exponential curve for calculating the write fees per kb to have smooth transition with bucketlist size

[anupsdf]

What problem does your feature solve?

Currently, the write fees per kb is a straight line that depends on current bucketlist size. When the slope of this line is steep, then fees can drop significantly lower when bucketlist size drops. We also have a target bucketlist size after which fees instantly jump by 1000 times.

What would you like to see?

Evaluate if having an exponential curve would solve these two problems. For example,

y = b ^ x + k
where, y = write fee per 1 kb
b = some base (we could start with natural base e which is 2.717828)
x = current bucketlist size
k = some constant

Here is a chart on how this would look like,

[MonsieurNicolas]

An exponential curve was something we actually considered before (even for the second part, passed the bucketListTargetSizeBytes as mentioned in the CAP), but in both cases, the "back pressure" from fees increasing just kicks in too slowly: from your chart you can see that most of the growth is happening when the ledger is getting very close to full.

[anupsdf]

One thought I have is that if we are going to revisit the fees more often, then in an exponential curve we can have the network settings such that current bucketlist size is 1 (or 2) GB away from when fees start to drastically increase. We then constantly adjust network settings and shift the curve as needed with bucketlist size increases.

[MonsieurNicolas]

when you "zoom in" on the last X bytes, exponential curve always have that property I mentioned above that price does not grow fast enough until it's too late -- one of the things we could consider is making MINIMUM_WRITE_FEE_PER_1KB a setting.

The thing to keep in mind is that the reason there is an explicit bucketListTargetSizeBytes is that this allows to reason about minimum hardware specs to run nodes, so whatever we do has to be parametrized around that.