New difficulty algorithm

[zawy12]

If you ever need to fork, here's a much better difficulty algorithm. This might be the best there is. If I find something better, I'll post it here.

[haruto-tanno]

Thanks @zawy12 I'll see if that'll be good solution (at least on lab). Currently, we are quite happy with the last algo (though it's not perfect) to find botnets, surge hashrate etc.

[zawy12]

I keep the newest version of the best algorithms like the one above here.

Masari copied your algo and had a lot of problems. They've done awesome with the WHM I helped devise and I'm getting them to go to EMA in their next fork.

The following shows your history compared to other coins, and how Masari is doing a lot better.

https://github.com/zawy12/difficulty-algorithms/issues/6

[zawy12]

There was an error in the way I was handling timestamps that's been corrected in the link to the EMA algorithm.

[zawy12]

We've developed a new algorithm that's the best of the best and allows integer math on the target.

https://github.com/zawy12/difficulty-algorithms/issues/17

[screab-idh]

Wouldn't this stay better in a pull request ?

[zawy12]

I don't have completed code for pull requests.

The algorithm that works the best is the LWMA aka WHM. Bitcoin gold is going to use it and they posted python code for it in the comments. They also found an error that's been corrected.

[zawy12]

Here's live data on 5 monero clones using LWMA

[zawy12]

Here's karbowanec's results after switching from an SMA N=17 like Sumokoin's to the new LWMA. Sumokoin is the only coin remaining (that I am aware of) that isn't about to upgrade from N=17 to the new one. Forknote will upgrade so that new coins created via forknote will not have the problem.

The N=17 was really fast, but it caused way too many problems due to varying up and down too much, inviting miners to constantly engage in on-off mining which you can easily see as the oscillations. The new N with LWMA is chosen to be as fast as possible without inviting these on-off attacks with accidentally low difficulty.

The chart shows a timestamp manipulation attack that occurred before the switch to LWMA (last two plots). The blue and magenta spikes are where the difficulty is not matching network hash rate closely. A difficulty algorithm's purpose is to minimize those.

[zawy12]

All of the coins who copied Sumo's difficulty algorithm are experiencing an attack like the following. It is not merely a hash attack like Sumo recently had, but they included bad timestamps. [edited to correct my factual error that guangvu3 mentions below:] Some time in the past, Sumo code corrected the vulnerability that is caused by Future_time_limit set to 7200. They also corrected the MTP variable. But their clones did not. Attackers assigning future timestamps causes the difficulty to drop up to 30% per block until it is zero. It only takes a 20% hash rate miner to perform the attack, so POW does not provide protection. Most of the coins that had Sumo's code have upgraded to LWMA, those who also fixed future_time_limit are no longer vulnerable.

IntenseCoin will have to revert the chain 5,000 blocks or more and fork.

[zawy12]

17 times I warned Sumo last year when we developed the algorithm that they really really needed to allow negative solvetimes (instead of sorting them).

[quangvu3]

Simply copying the code without knowing the inside settings is problem. You cannot attack Sumo by timestamp (unless you have 80% or more hashpower and very accurately) bz Sumo allows only 30 min ahead (not behind) and any wrong timestamp forward would be neutralized by next correct block due to internal sort.

[zawy12]

30 minutes ahead of previous block? That's not real protection with the future_time_limit still set to 7200 if the attacker is > 50%. Since BLOCKCHAIN_TIMESTAMP_CHECK_WINDOW is still 60, if they haven't also limited reverse times like I recommended last summer, all an attacker needs to do is send -30x240 timestamps once every 3 blocks and difficulty will go to zero shortly... which is a direct result of using the sort.

There's no neutralizing because there are no negative solvetimes being allowed to follow bad timestamps because of the sort.

[quangvu3]

Oh, you looked at wrong places, Sumokoin uses these:

#define CRYPTONOTE_BLOCK_FUTURE_TIME_LIMIT_V2           60*24
#define BLOCKCHAIN_TIMESTAMP_CHECK_WINDOW_V2            12

Actually, you can set ahead 24 mins only and pls note that 6 first blocks (equal to 24 mins block time) are always cut from diff calculation, so to successfully attack timestamp you'll need to sneak into full 6 blocks with forward timestamp. As discussed before, it'll be very tough job even IF you have majority of hashpower.

30 minutes ahead of previous block?

No, 30 mins from current (server) time, it makes the job much tougher

And that's why, through many kinds of attacks, the algo is standing. I don't say it's perfect but it has no major flaws for sumokoin. Copying code won't work unless they have the same settings, the same blocktime etc.

[zawy12]

Oh, sorry, I was mistaken. That should be good protection from bad timestamps.

[sumoprojects]

-- Resolved