ivy: add @ operator modifier

[rsc]

The @ operator modifier is a bit like APL's each/map. Applied to the left or right of an operator, it produces an operator that applies to each element of its left or right argument and then returns an array of those results.

For example consider a vector of x,y points and the operator computing the Manhattan distance between two points:

d = 3 2 rho iota 6

op x dist y = +/ abs x-y

0 0 dist d
3 7 11

What if we want the Manhattan distance between all pairs of points? This is a kind of outer product but only matching pairs of points; the actual outer product matches all pairs of numbers.

The outer product does 'for all' over all dimensions of its left and right arguments, while @ only does 'for all' over a single dimension of the argument on the side where the @ appears. So the pairwise Manhattan distances of the points is @dist@:

d @dist@ d
0 4 8
4 0 4
8 4 0

What if we have a binary operator and want to apply it with a fixed left argument to each of a list of left or right arguments? In this example, because Manhattan distance only uses +/, abs, and -, and those automatically replicate a vector to match with a matrix, it works to say:

0 0 dist d
3 7 1

d dist 1 1
1 5 9

but in general it would not. In the general case, dist@ and @dist work:

0 0 dist@ d
3 7 11

d @dist 1 1
1 5 9

What if we have an operator to compute pairwise distances of a list of points and want to apply it to a list of point sets? Then we can use a single @ on the right (for a unary operator, it would never make sense to use @ on the left):

op dists d = d @dist@ d

dx = 4 3 2 rho iota 24

dists@ dx
0 4 8
4 0 4
8 4 0

0 4 8
4 0 4
8 4 0

0 4 8
4 0 4
8 4 0

0 4 8
4 0 4
8 4 0

Outer product can be viewed as a special case of @. Specifically, x o.f y is x @...@f@..@ where there are (rho rho x) @s on the left and (rho rho y) @s on the right.

(2 rho iota 3) o.^ 2 2 rho iota 4
0 3
2 5

3 0
1 6

(2 rho iota 3) @^@@ 2 2 rho iota 4
0 3
2 5

3 0
1 6

[rsc]

Not for review. To support conversation in #69.

[robpike]

I'd need to see the documentation.

[tcolgate]

If we swtiched @ to each and added docs, (in line with the discussion in #69), would this PR be acceptable?

[bear8642]

These seem fairly similar to K's each-left and each-right, which you can model in APL using the rank(⍤) operator. Were you aware of those when you were designing these?

[robpike]

The rank operator applies to "functions" as a first-class data type, which Ivy does not have. I'd like it to have them but it's a big change internally and I've never managed to make it happen. It may happen one day. I still think about it.