search

clab / dynet

DyNet: The Dynamic Neural Network Toolkit
Apache License 2.0
3.41k stars 707 forks source link

High-order tensor multiplication #942

Closed pcyin closed 6 years ago

pcyin commented 6 years ago

Right now we don't support multiplications of tensors whose order is higher than two. An MWE:

W = model.add_parameters((7, 5))
X = dy.inputTensor(np.random.rand(5, 4, 3, 3), batched=True)
o = dy.parameter(W) * X
print o.dim() # ((7, 4), 3), should be ((7, 4, 3), 3)
neubig commented 6 years ago

Inconsistent behavior fixed by https://github.com/clab/dynet/commit/b94f60334f6265ff9e13540a76c8827659320492

The underlying problem that we can't multiply matrices of higher order is somewhat covered by an open issue on general-purpose tensor contractions: https://github.com/clab/dynet/issues/91

(In the particular case of multiplying the last dimension of the first matrix by the first dimension of the second matrix, we can solve this by doing clever reshaping within the existing MatrixMultiply code, which also might be worth doing, as it will be more efficient than the general case.)