search

google-research / dex-lang

Research language for array processing in the Haskell/ML family
BSD 3-Clause "New" or "Revised" License
1.56k stars 106 forks source link

Replace floating point sizing formula in stack with integer operations. #1321

Closed duvenaud closed 1 year ago

duvenaud commented 1 year ago

The required code was already in the FFT demo.

duvenaud commented 1 year ago

Not sure if this could be a test, but to put your mind at ease, here are the old and new functions:

for req_size:(Fin 17). f_to_n $ 2.0 `pow` (ceil $ log2 $ n_to_f (ordinal req_size))
[0, 1, 2, 4, 4, 8, 8, 8, 8, 16, 16, 16, 16, 16, 16, 16, 16]

for req_size:(Fin 17). intpow2 $ nextpow2 (ordinal req_size)
[1, 1, 2, 4, 4, 8, 8, 8, 8, 16, 16, 16, 16, 16, 16, 16, 16]