AviSynth / AviSynthPlus

AviSynth with improvements
http://avs-plus.net
963 stars 73 forks source link

FMod precision #394

Open flossy83 opened 5 months ago

flossy83 commented 5 months ago

According to this it's expected behaviour, and I tested it in C++ and got the same result as below:

string = 
\ "FMod(0.0, 0.1): " + String(FMod(0.0, 0.1)) + " (0.0 / 0.1 = " + String(0.0/0.1) + ")\n" +
\ "FMod(0.1, 0.1): " + String(FMod(0.1, 0.1)) + " (0.1 / 0.1 = " + String(0.1/0.1) + ")\n" +
\ "FMod(0.2, 0.1): " + String(FMod(0.2, 0.1)) + " (0.2 / 0.1 = " + String(0.2/0.1) + ")\n" +
\ "FMod(0.3, 0.1): " + String(FMod(0.3, 0.1)) + " (0.3 / 0.1 = " + String(0.3/0.1) + ")\n" +
\ "FMod(0.4, 0.1): " + String(FMod(0.4, 0.1)) + " (0.4 / 0.1 = " + String(0.4/0.1) + ")\n" +
\ "FMod(0.5, 0.1): " + String(FMod(0.5, 0.1)) + " (0.5 / 0.1 = " + String(0.5/0.1) + ")\n" +
\ "FMod(0.6, 0.1): " + String(FMod(0.6, 0.1)) + " (0.6 / 0.1 = " + String(0.6/0.1) + ")\n" +
\ "FMod(0.7, 0.1): " + String(FMod(0.7, 0.1)) + " (0.7 / 0.1 = " + String(0.7/0.1) + ")\n" +
\ "FMod(0.8, 0.1): " + String(FMod(0.8, 0.1)) + " (0.8 / 0.1 = " + String(0.8/0.1) + ")\n" +
\ "FMod(0.9, 0.1): " + String(FMod(0.9, 0.1)) + " (0.9 / 0.1 = " + String(0.9/0.1) + ")\n" +
\ "FMod(1.0, 0.1): " + String(FMod(1.0, 0.1)) + " (1.0 / 0.1 = " + String(1.0/0.1) + ")"

BlankClip.Text(string, lsp=0)

debug avs_snapshot_00 01 079

What's bugging me though is the values on the right side seem to imply the FMod result should have been 0, but obviously the precision internally is different than what is being shown.

And perhaps the bigger issue is why is the remainder 0.1 - shouldn't it be something really tiny like 1.49012e-08 or something?

I acknowledge that Avisynth is not doing anything technically wrong here - it's just returning whatever C++ says the FMod value is. Which kind of makes it useless - how are we supposed to do a check to ignore really tiny remainders if we're getting such a huge remainder on ones like FMod(0.5,0.1)?