small-precision division

Co-dfns / mystika

High-end Cryptographic Library

GNU Affero General Public License v3.0

43 stars 26 forks source link

small-precision division #60

Open llelf opened 4 years ago

llelf commented 4 years ago

⍝ div_tests
DIV∆29_TEST←{_←X  16 1 0 8 ⋄ M.div 16 0 0 2}
DIV∆30_TEST←{_←X  16 0 0 1 ⋄ M.div 16 0 0 1}

Shouldn’t it be like this? (Currently both crashing)

Tikhon03 commented 4 years ago

That is a peculiarity with the length, since

DIV∆29TEST←{←X 16 1 0 0 8 ⋄ M.div 16 0 0 0 2} DIV∆30TEST←{←X 16 0 0 0 1 ⋄ M.div 16 0 0 0 1}

both would pass. I will try to fix it soon, but I generally am focused on having it work for numbers that are actually big, which in base 16 means there should be more than 8 places.

Tikhon03 commented 4 years ago

The current version in my repo has a fix. Once arcfide accepts the pull request it will be available in the codfns repo. Generally speaking when testing extreme cases I pay the most attention to when the number of places is maxed out (see the mul tests for example). At the low end I always check that it works for however many places give 32 bit precision (in various bases). The example you gave is only 4 bit precision.