2 bugs in mirostat sampler

[cahya-wirawan]

Hi,

1. The first bug is in the function estimate_s. The function is unstable because the probability in denominator could be 0.0. As result, num below become NaN.

2. The second bug is in the function compute_k. This function is very unstable, the result is overflow.

• n.powf(-eps) = 1 / n.powf(eps) -> for n=65536, it can become overflow easily
• powf(1.0 / s) -> s can become 0.0 -> unstable.

A plot of the formula *k = (eps 2.0_f32.powf(tau) / (1.0 - n.powf(-eps))).powf(1.0 / s)** using wolframalpha show its stability:

[cryscan]

The power in the formula is n^(-eps) so it should be 65536^(1-x), and this is what I get:

And yes, estimate_s is unstable.

[cryscan]

Just pushed a fix. would you like to try again?

[cahya-wirawan]

yes, may bad, I overlook -eps. I tried the fix, it runs now without overflow so far. However, the formula is in my opinion unstable because k is undefined for s==1.0 and s==0. k is also huge or overflow for -0.5<s<0.