I thought I was completely finished studying difficulty, but important things keep coming up. The previous interesting question I didn't expect was how to do a difficulty algorithm for DAGs. I always thought there wasn't much one could do in difficulty and it was just a game (puzzle) to find the perfect one. I expected it to be only marginally better than simple moving average, and that's totally true for large coins, as BCH's new algo has shown. But I was totally wrong as far as it only being a mostly-pointless game. It's turned out that a good knowledge has been crucial in solving important problems i DAGs and VDFs (which are the future), not to mention quickly seeing when teams much smarter than me (like Chia & Ethereum) mess something up. Or seeing when an important figure like "MF" has a scheme that appears super-intelligent and yet is really awful once the wool is pulled back, making me think certain BIPs in BTC need to be rejected until a different big name is behind it. I mean we mortals have to depend on the anti-scientific appeal to authority, aka the centralization of dev reputation, unless and until crypto is greatly simplified.
Chia's difficulty algorithm will be a disaster if their farmers are like POW miners who can come and go in response to difficulty changes. By having a 504 block delay after the difficulty calculation before the adjustment is applied, they will have a positive feedback loop in miners jumping on and off.
This is probably what caused all small coins using Cryptonote/Monero algorithm to blow up after a few months, getting the chains stuck as a result of cut and lag delays that were a LOT less.
Ethereum's difficulty algorithm was almost a really good. The max() function prevents it from being D = prevD*(1+1/N - t/T/N) which is the EMA with the e^x=~ 1+x substitution.
The simplest difficulty algorithm is really interesting and turns out to be required to fix VDF problems. The simplest algorithm adjusts difficulty up or down a specific percentage like 1% in each block if previous solvetime was above or below target solvetime (or the median ln(2)*T). I didn't post an article in the past because it wasn't needed, but it was interesting in how it can simulate other difficulty algos with long averaging windows. I need it now to fix Chia's (and my VDF-POS scheme's) problem of difficulty being grindable (affecting the randomness of leader selection) which can increase an attacker's chances of winning blocks.
The "perfect" difficulty algorithm. Using Erlang (or gamma) distribution to design the "perfect" difficulty algorithm. The EMA is almost perfect, but there's still some error at low N. By "perfect" I mean it should be able to make a 100% change in the difficulty based only on the previous block without any "buffering" or "tempering" that seems to be always required. It's the perfect algorithm (adjustment) from a theoretical perspective (as far as I can tell) if the mean and median solvetimes under constant hashrate are T and T*ln(2) respectively. If you see only 1 instance of a delay and know it's a Poisson process, should you assume you just saw the mean or the median time? There's a 50% chance that your sample was only ln(2)=0.693 of the mean time. Given the "perfect" algorithm for a single sample, it should be possible to build algorithms that achieve any other objective, such as achieving the fastest response to hashrate change for a given level of stability under constant hashrate (aka "best response*stability factor")
I thought I was completely finished studying difficulty, but important things keep coming up. The previous interesting question I didn't expect was how to do a difficulty algorithm for DAGs. I always thought there wasn't much one could do in difficulty and it was just a game (puzzle) to find the perfect one. I expected it to be only marginally better than simple moving average, and that's totally true for large coins, as BCH's new algo has shown. But I was totally wrong as far as it only being a mostly-pointless game. It's turned out that a good knowledge has been crucial in solving important problems i DAGs and VDFs (which are the future), not to mention quickly seeing when teams much smarter than me (like Chia & Ethereum) mess something up. Or seeing when an important figure like "MF" has a scheme that appears super-intelligent and yet is really awful once the wool is pulled back, making me think certain BIPs in BTC need to be rejected until a different big name is behind it. I mean we mortals have to depend on the anti-scientific appeal to authority, aka the centralization of dev reputation, unless and until crypto is greatly simplified.