Idea: Typed axes for array variables and parameters

[tkf]

When there are multiple array variables and parameters in the model you have, you need to match the sizes of them. Rather than manually verifying the consistency of them, a better approach would be to encode the semantics of each "axis" in the computation graph. It enables SciCompDSL.jl to catch some errors at graph creation time.

Example (from my previous comment: https://github.com/JuliaDiffEq/DifferentialEquations.jl/issues/261#issuecomment-368772516):

a = ArrayAxis(:a)
b = ArrayAxis(:b)
xa = DependentVariable(:xa, a)
xb = DependentVariable(:xb, b)

# Parameters are block matrices
Maa = Parameter(:Maa, a, a)  # Na by Na matrix
Mab = Parameter(:Mab, a, b)  # Na by Nb matrix
Mbb = Parameter(:Mbb, b, b)  # Nb by Nb matrix
τa = Parameter(:τa, a)     # Na-dimensional vector
τb = Parameter(:τb, b)     # Nb-dimensional vector

de = @diffeq begin
  D*xa ./ τa = -xa .+ tanh.(Maa * xa .+ Mab * xb)
  D*xb ./ τb = -xb .+ tanh.(Mbb * xb)
end

where Na = size(Mab, 1) and Nb = size(Mab, 2) when Mab is a normal array. It is, as a whole, Na + Nb-dimensional ODE.

Possible benefits:

• Imagine that you happen to have test code with Na == Nb (that's not a great test but I suppose it can happen when there are many axes). Then your code will run with Mab replaced by Maa and it takes some extra time to debug it.
• When the numeric arrays are passed to the problem constructor as the parameter values, constructor can check the consistency of them before running the graph. In the above example, e.g., size(Maa, 1) == size(Mab, 1) has to be satisfied.

Related project:

• Intel® Nervana™ graph (a deep learning library) https://ngraph.nervanasys.com/docs/latest/axes.html

[tkf]

There is a discussion on adding "Named Dimensions" to Open Neural Network Exchange:

• https://github.com/onnx/onnx/issues/406
• https://github.com/onnx/onnx/pull/443

But probably the discussion is too much deep net-oriented.

[ChrisRackauckas]

We essentially have this now!