Closed danieldk closed 2 years ago
Example of undeterministic behavior:
from thinc.api import NumpyOps
import numpy as np
ops = NumpyOps()
a = np.ones((20, 30), dtype="float32")
b = np.zeros((0, 20), dtype="float32")
for i in range(3):
print(ops.gemm(np.copy(a), np.copy(b))[0][:10])
$ python test.py
[ 7.2064716e+31 1.8474847e+31 1.3043222e-11 6.4460709e-10
7.3712094e+28 7.2697323e+31 -2.9495326e+12 4.6179864e+24
7.9981108e+20 nan]
[-1.8212505e+38 7.1954651e+28 9.1228076e+31 1.1784816e+35
3.2676755e+33 3.4466046e+32 7.8751016e+34 -1.3883587e+33
1.0641391e+32 5.0848063e+31]
[ nan -3.6532688e+33 1.8077397e+28 nan
1.3647307e+34 2.5540607e+30 nan -2.0071014e+38
-1.9679860e+38 nan]
Found while investigating https://github.com/explosion/spaCy/issues/10107 with @adrianeboyd .
I mainly wondered whether there was a really standard error message used for this, but there doesn't seem to be.
gemm
incorrectly allowed multiplication of two matrices that do not have the same inner sizes K:(N, K) x (K, M)
This leads to reads of uninitialized memory and non-deterministic results.