Elementwise binary operations with matrix and scalar

[daphne-eu]

In GitLab by @pdamme on Mar 1, 2022, 19:29

In linear algebra programs, elementwise binary operations where one input is a matrix and the other input is a scalar are commonplace. When mixing matrices and scalars, currently the DAPHNE system only supports a matrix on the left-hand side and a scalar on the right-hand side, whereby the user must take care to adhere to this order when writing algorithms in DaphneDSL, DAPHNE's domain-specific language. This requirement shall be relaxed.

The task is twofold: On the one hand, we need canonicalization rewrites for DAPHNE's EwBinaryOp which swap the left-hand side and right-hand side input to ensure matrix-scalar operations, for commutative operations. On the other hand, non-commutative operations shall be handled by a dedicated kernel for scalar-matrix operations (which shall be an adaptation of the existing matrix-scalar kernel). Optionally, non-commutative operations may also be handled by a compiler pass adapting the operands and the operation, e.g., 1 – mat to mat + -1, but note that this is not possible for all operations.

[RetoKrummenacher]

Given the fact that left scalar multiplication is not supported yet, the example in the DaphneDSL documentation should be changed from 1.5 * X @ y + 0.001 to X @ y * 1.5 + 0.001 as the first is currently not working with X being a matrix and y a column vector.

X = fill(1.0,10,10);
y = fill(2.0,10,1);

# not working
r = 1.5 * X @ y + 0.001;
# Output:
# JIT session error: Symbols not found: 

# working
r = X @ y * 1.5 + 0.001;
print(r); 

[pdamme]

Commit 1033729696a362739f56cdba8d774ca70573318c just introduced scalar-matrix ops for +, -, *, / via rewrites to matrix-scalar ops that use the existing kernels. The commit message may be worth reading for someone working on this issue. However, this is just a quick fix, the issue needs to be addressed more systematically.