New difficulty algorithm

[zawy12]

Here is my recommendation for your new difficulty algorithm. You can use either one because they act very similar, but I have a preference for the first one.

[zawy12]

This is another adjustable window method. Gives a smooth difficulty most of the time, and triggers on unusual events.

Dynamic EMA

# Dynamic EMA difficulty algo (Jacob Eliosoff's EMA and Zawy's adjustable window). 
# Bitcoin cash dev(?) came up with the median of three to reduce timestamp errors.
# For EMA origins see 
# https://en.wikipedia.org/wiki/Moving_average#Application_to_measuring_computer_performance
# "Dynamic" means it triggers to a faster-responding value for N if a substantial change in hashrate
# is detected. It increases from that event back to Nmax

Nmax=144 # max EMA window
Nmin=25 # min EMA window

# TS=timestamp, T=600
# Find the most recent unusual 20-block event
for (i=height-Nmax to height) {  # height=current block index
    if ( (median(TS[i],TS[i-1],TS[i-2]) - median(TS[i-20],TS[i-21],TS[i-22]))/20/T  > 1.65 
           or  
         (median(TS[i],TS[i-1],TS[i-2]) - median(TS[i-20],TS[i-21],TS[i-22]))/20/T  < 0.5  ) 
      {   unusual_event=height - i + Nmin   } 
}
N = min(Nmax, unusual_event))

# now do EMA in previous post with this N

[zawy12]

Just for simplicity reasons, I think Jacob Eliosoff's simple EMA above with N=80 is the one to use. I am making it slower than N=17 because I am seeing miners increase 3x if price/difficulty ratio drops 30% and N=17 causes it to drop that much on accident. N=17 is causing more delays in your coin than it needs to be. Right now my measure of your delays is 3.5% and I think this new algo will get it down to 1% and the difficulty will be a lot smoother. I believe its response to sudden hashrate change will be good.

[aivve]

I will try to understand and implement this one. This is amazing work.

[zawy12]

I keep waffling between simple EMA and the dynamic EMA. I am still working to try to improve it.

[zawy12]

I had en error in the dynamic one that I've corrected.

[zawy12]

A word of caution with the EMA: when converting to and from bits field (aka target) to difficulty, it is important to not make a consistently wrong round off error. For example, if previous difficulty was 100,000, it is important that nothing in the code make it consistently +50 more or consistently -50 less (0.05% error). That would cause the EMA at N=70 to have 3.5% error in solvetime. At 0.5% error per block, there is 35% error in the solvetimes (difficulty is 30% too high or low). The error that develops from 0.5% and using N=70 seems to be based on about 1.005^70 = 41%. Larger N means larger error from "round off". If half the time it is +1,000 too high and the other half -1,000 too low, then it's OK. just don't be consistently wrong in the same direction.

[zawy12]

A bad exploit on the timestamps was possible on the EMA, so I corrected it above. An attacker could have just sent -6xT timestamps which are made T/100 solvetimes, but then an honest timestamp after it would be about 7xT which would drive difficulty down. So ironically an attack pretend to send difficulty up, but makes honest timestamps send it down.

The more complicated EMA is not ready, and may never be clearly better, so use the simple EMA I have above.

[zawy12]

I keep the newest version of the algorithm here.

Masari has done awesome with the WHM and I'm getting them to go to EMA in their next fork. Masaris used Sumokoin's N=17 and it was terrible. This like below shows their history, but that page also shows your history.

https://github.com/zawy12/difficulty-algorithms/issues/6#issuecomment-352468230

Oh, I forgot, I don't have your history because I can't get your json working like I did with the other monero coins. How can I correct this?

curl -X POST http://127.0.0.1:32348/json_rpc -d '{"jsonrpc":"2.0","id":"0","method":"getblockcount"}' -H 'Content-Type: application/json'

{"error":{"code":-32603,"message":"JsonValue type is not ARRAY or OBJECT"},"id":"0","jsonrpc":"2.0"}

[zawy12]

I had an error in the timestamp handling in the EMA I posted above and in my link. Here it is corrected. which has a new recommendation for N (I did some work trying to select the best N based on the T)

[aivve]

Please, try this request

{
  "params": {

  },
  "jsonrpc": "2.0",
  "id": "test",
  "method": "getblockcount"
}

[zawy12]

That worked for "test" and "0", but now I can't get this to return a block. Give "method not found". I'm trying to get height, timestamp, and difficulty for each block.

curl -s -X POST http://127.0.0.1:32348/json_rpc 
-d '{"params":{"height":1000} ,"jsonrpc":"2.0","id":"0","method":"getblock"}' 
-H 'Content-Type: application/json'

[zawy12]

OK, I found your RPC documentation and got it working

[aivve]

I should take time and add more methods to documentation. Forknote methods are working for Karbo too.

[zawy12]

I added karbowanec to the charts: https://github.com/zawy12/difficulty-algorithms/issues/6#issuecomment-352900192

[zawy12]

OK, here's what's hopefully my final algorithm. It works as good as all the best ones, but it also allows integer math that coins want and allows negative solvetimes which makes it more accurate and safe with bad timestamps.

https://github.com/zawy12/difficulty-algorithms/issues/17

[ghost]

Bytecoin Issue related to the difficulty adjustment and time-warp attacks.

[zawy12]

I'm closing this because I've switched to a new difficulty algorithm which makes all the above irrelevant.