Integer overflow

[milankl]

The way you implement stochastic rounding here

https://github.com/cboutsikas/stoch_rounding_iplicit_reg/blob/da3a000e18182393596918697e9e3c00ff197250/nu_randn_10e4_all.jl#L101-L114

you can easily hit an integer overflow

julia> SR_fix_dec(π,16)
3.141592653589793

julia> SR_fix_dec(π,17)
3.1415926535897936

julia> SR_fix_dec(π,18)
3.1415926535897936

julia> SR_fix_dec(π,19)
-2.6536229128719047

julia> 10^19
-8446744073709551616

julia> 10^18
1000000000000000000

[cboutsikas]

Currently, we test on fixed number of decimal digits, namely for dec_po = 1:5. But, this is actually a good point, thanks for pointing this out.

[milankl]

Yeah, for 1:5 you're probably in the safe zone, but just pointing this out because that's not how stochastic rounding would be implemented in hardware where it's really about stochastically flipping the last bit (and probably some carry bits) of a given float. With this approach you run into problems, (if I understood this correctly in a quick check)

julia> SR_fix_dec(1e16,1)
1.0e16

would never be stochastically rounded even though a perturbation on the 1. decimal place is perfectly representable

julia> nextfloat(1e16)
1.0000000000000002e16

The reason being that it's a number above

julia> maxintfloat(Float64)
9.007199254740992e15

so your floor operation doesn't do anything because a*10^dec_pos is already rounded to integers (no non-integers are representable beyond maxintfloat)

julia> maxintfloat(Float64)
9.007199254740992e15

julia> maxintfloat(Float64)+π   # equivalent to adding pi=4
9.007199254740996e15

and also the fl_a += 1 operation is equivalent to adding zero

julia> maxintfloat(Float64)
9.007199254740992e15

julia> maxintfloat(Float64)+1
9.007199254740992e15

[cboutsikas]

I forgot to mention that we test on numbers being in the range [-1,1]. So, we do not have to deal with numbers having these kind of large exponents. Regardless, you are making some good points and that might come in handy in case we want to change the setting of our experiements.