Closed axch closed 1 year ago
Added some measure of consistency testing between run-time and compile-time casts. Unfortunately, it's hard to test many values, because compiling and running a million 1-instruction LLVM programs is much more expensive than executing a million LLVM instructions. That said, the tests did reveal two bugs (one already present and one introduced by me in the previous commit), which fixed.
Caveat: Did not check that they agree bit-for-bit with the runtime behavior, but any discrepancies should be small.
Benefit:
This ends up constant-folding the cast from the literal zero to the relevant floating-point type in the
Add
instance for floats.Therefore, that zero ends up being an
Atom
and not an expression when we build theBaseMonoid
for adding floats, and so is inlined therein.Therefore, that zero is, in effect, rematerialized when doing AD.
Therefore, an n by m by k tensor of those zeros is not stored by linearize of matmul, so the jvp-matmul benchmark runs some 30% faster. Of course, it is still storing two other tensors it shouldn't store, but that's for another time.