Support basic function introspection and invocation

[arteymix]

@rainwoodman

We will have to define a set of GIR attributes to use across the generated code, all in the Vast.* namespace.

Here's a few:

• gradiant
• gradiant_cname (to override default vast_ prefix)

The list will grow up as we implement the codegen.

[arteymix]

We might need an expression parser to do stuff like sin -> -cos.

[rainwoodman]

@ expression parse: no. I don't think we want to do that -- a parser is a gate to hell.

I'd just define a new function called gradient_sin which calls cos and then multiply by negative one, which is more flexible and general.

@ I think we need named function arguments -- in order to describe how argumenhts of gradient and objective functions are related e.g.

c, d = f(a, b)
g_a, g_b = fprime(a, b, c, d)
``
a b c d of fprime are related to a b c d of f.

Keeping track of these with an integer(array index) will be welcoming errors.

[rainwoodman]

I think we also need another member that allocates the output based on the input -- take a look at the Graph pull request (see the test case) -- https://github.com/rainwoodman/vast/pull/18

This is needed as we evaluate a graph -- the temporary variables shall be allocated automatically.

[arteymix]

Allocation will become tricky if we decided to make Array an interface: which kind of array to allocate? This is why I think we should push on having the caller to allocate.

In the case of evaluating a graph, the user has to somehow tell the graph how/where to allocate temporary array and preferably not rely on that at all by reusing existing ones.

Thus, we should move that logic into the graph if possible so that we can work on arbitrary Array and Function objects.

[arteymix]

I agree, it's a good idea to introduce intermediary function than having to parse expressions.

Arguments name in function declarations are introspected. I'm working on a to_string utility for Function at this moment.

I'll look into allocation mechanism as well, we really need to make that piece pluggable into the network.

[rainwoodman]

No. allocation is not an issue as it seems.

If we are dealing with the array pointers we already knew what memory is for this type of device -- which means for a different type of device we will need a different set of Function objects anyways, and they can allocate the memory differently.

[arteymix]

We definately need to have expression for describing partial derivatives of our mathematical functions.

d_x power (x, a) -> if (ge (a, 0), multiply (a, power (x, a - 1)), 0)

We don't really need to type-check anything, just generate a graph and it should be okay if there is a transformation defined for the given types between operations

The other thing is that since we support multiple output values, we will have to have partial derivatives for all pair of input/output variables. I think we should put the annotation directly on the output variables.

public
sincos (Array x, [Vast (gradient_x = "cos (x)"] Array y, 
                 [Vast (gradient_x = "negative (sin (x))"] Array z)

We can use https://valadoc.org/glib-2.0/GLib.Scanner.html to scan parenthesis, identifiers and constants in a recursive descent. Nothing too complicated because we have one production in the grammar.

Thus I'll remove the gradient-related code so that we can merge function introspection quickly. We will work the whole thing along with the computation graph implementation.

[arteymix]

Okay, finally, we can define functions for the partial derivatives, but we will have to introduce a Gradiant class to do the logic of retrieving these functions and evaluating it on a point.

public void
power (Array x, Array y, Array z);

public void
power_gradient_dz_dx (Array x, Array y, Array z); // compute z = a * power (x, a - 1)

public void
power_gradient_dz_dy (Array x, Array y, Array z); // compute z = power (x, a) * ln (a)

[arteymix]

@rainwoodman any reviews before we get this merged?

I do not see the necessity of doing in/out direction for arguments and is seems sufficient for now to just consider function arguments as both in and out. Nonetheless, we could add [Vast (direction = "in")] annotations later.

Also, this is only covering element-wise operators. We will probably want a way of telling along which axis performing reduce-like operations.

[arteymix]

We might want to look into named arguments for anything beyond input and output arrays. While input and output arrays are typically well known, we probably want some flexibility for adding new arguments without breaking ABI.

public unowned Array
sum (Array x, Array z, ...);

Vast.sum (arr, arr, axis: 1);

We also have to add assertions for function that does not support in-place evaluation (e.g. most non-element-wise stuff).

[arteymix]

@rainwoodman I'll merge this so that you can work on the computation graph

You should seriously consider the slice allocator for allocating temporary arrays. I think it's pretty common to require equal-sized chunk of bytes when evaluating an expression. Basically, that would allow you never to have to worry about keeping track of pre-allocated temporary arrays.