Difficulty prediction doesn't take into consideration the difficulty bomb.

[mathvdh]

Hi Everyone !

First : A big thank you for this nice tool and for making it open-source (I was thinking about coding something similar, but I'd better collaborate on this one)

👍

After quickling scanning the code of predicting-difficulty/prediction-service.js

I couldn't find anything related to the difficulty bomb. I believe this should be taken into consideration ... Here are some relevant infos about it :

http://ethereum.stackexchange.com/questions/3779/when-will-the-difficulty-bomb-make-mining-impossible/3800

https://www.reddit.com/r/ethereum/comments/3jynkc/difficulty_adjustment_scheme_block_200000_coming/

https://github.com/ethereum/EIPs/blob/master/EIPS/eip-2.mediawiki

Let me know what you think about it ?

Thanks

[mathvdh]

There is also this script to calculate it roughly

def f(diff, n):
    """
    This function returns the new difficulty.
    """
    period_count = (n + 1) // 100000
    return  max(diff + 2**(period_count - 1), 5000)

diff =  6232935860523 # starting diffculty
blocknumber = 200000 # starting blocknumber
numberOfBlocks2years = 2*365*24*60*60/15
for x in range(0, numberOfBlocks2years):
    blocknumber = blocknumber + 1
    diff = f(diff, blocknumber)
    if (blocknumber % (numberOfBlocks2years/24)) == 0:
        blocktime = 1.0 * diff / 400000000000 #hashrate
        print "Diff after ",str((blocknumber-200000.0) / (numberOfBlocks2years/24)), " month is: ", str(diff), "and avg block time is: ", str(blocktime)

Which I found here : https://gist.githubusercontent.com/CJentzsch/c78768f9837afb8eef74/raw/618bf4cd84a16ebb4fa5c514e6584c939b216769/gistfile1.txt

[mathvdh]

Basically what it means (Or what I understand from it).

It's that the difficulty behave linearly at the moment but when the time bomb will start, it will start behaving exponentially and start exploding.

On the other end, I guess that there will be a hard-fork before the 'explosion' otherwise, the blocktime will become huge and the network unusable ...

[AgeManning]

Hey @mathvdh Thanks for the interest and comments.

I know about the difficulty bomb, specifically equation 44 (page 6) on the ethereum yellow paper (http://gavwood.com/paper.pdf).

Its an exponential growth factor hard-coded into the protocol and grows like exp^{BN/100000 -2}. where BN is block number.

A friend of mine specifically brought up this question and asked why I hadn't included it. I remember taking about 30 mins to convince him that it wasn't necessary, so hopefully I can do it in text here. The justification of its non-inclusion is this:

The predictive difficulty is simply a least squares best fit estimate. It samples past difficulty and fits a line given that data. The uncertainty, or unknown element of this is simply network hashrate, which is what the curve is predicting. Given any known arbitrary function governing difficulty, it has no uncertainty and so will have no effect to the least squares estimate. Let me illustrate with some examples.

Consider a difficulty which increases linearly (not exponentially like the difficulty bomb). Lets say it has the simple equation: D = 2*d, where d is number of days, D is difficulty. And we wanted to predict the next 30 days.

If there were no miners at all, D at 30 days would be 60. If I did not include the D=2d function anywhere in the estimate, my algorithm would look at day 1 see D = 2, it would look at day 3 and see D=6. and would fit a line and say at 30 D = 60, which is the correct answer (even though I didn't use the known function for difficulty in its calculation).

When there are miners, there are fluctuations, but the fluctuations will be around the D=2_d line, and my best fit will still go like D=2_d. The reason is that the function is known and contains no uncertainty and just sampling past points give all the information, including the known function. So we have no need to specifically include it in the prediction.

With the difficulty bomb, we just have an exponential. In time, the difficulty will grow but not because of the hashrate. Either way it is irrelevant for calculating future difficulty because sampling past points will already account for this exponential growth.

Hopefully this makes some sort of sense. Ask more questions if this still isn't clear.

Hope it helps :)

[AgeManning]

The point of this exponential in the protocol, is not so that we hard fork before it happens. Its designed so that mining becomes very difficulty and rewards diminish. It is intended to force people to move to a PoS system rather than a PoW.

[mathvdh]

Ok, I get it. It makes sense ...

The only way I can see it being wrong is if you do like 6-12 month predictions before the time bomb start acting and then the behaviour changes from something linear to something exponential ..

BUT this being said, I think any 6-12 month predictions will be quite "unreliable" ...

About the reason of the timebomb, I understood it's to force the miners to transit to POS. But what I meant is that, if POS is not ready by then, a HF will be needed in order to correct the timebomb otherwise the network will become unusable ..

Anyway, thanks :)

[mathvdh]

Well I guess we can close this issue in any case ... I will open another issue to discuss another idea I have :)

[AgeManning]

Yep, with the prediction, its always about the timescale of sampling points vs how far into the future you want to predict.

The exponential is happening now. But if you only sample 100 days, its effect is negligible, so the fit will not see it and hence if you predict 2 years into the future, you will for sure have an unreliable prediction.

If however you could sample of 10 years and predict 1 year in the future, this exponential trend would be apparent, and the fit would become exponential and give a reliable estimate for the difficulty bomb.

For the prediction, as a rough estimate you should only predict 30% of the sampling distance. If you notice I write a little warning in the calculator about this.

So if you sample 100 days in the past, i recommend only predicting about 30 days into the future. :)

[mathvdh]

Yep it makes sense , you cannot predict 10 years in the future with only 1 year of data for example

I guess that would be called 'guessing' not predicting :)

👍