Vectorized operator application

[primo-ppcg]

This is a follow-up to this discussion.

Generally, I think it would be a very bad idea to rewrite all arithmetic operators to automatically vectorize if one or more of their arguments is a tuple or array. However, I think a function that vectorizes an operator could be a very nice language addition.

It can also be implemented fairly concisely:

(defn vec
  ``Vectorizes `op` onto `inds`. Scalars will be applied to each item of an indexed type, and
  indexed types will with operated upon pair-wise, with the shorter being padded with the operator
  identity.``
  [op & inds]
  (var len 0)
  (def ninds (length inds))
  (def iters (array/new-filled ninds))
  (loop [[k ind] :pairs inds :when (indexed? ind)]
    (set len (max len (length ind)))
    (put iters k true))
  (if (> len 0)
    (do
      (def res (array/new-filled len))
      (def call-buffer (array/new-filled ninds))
      (forv i 0 len
        (eachp [k ind] inds
          (put call-buffer k (if (in iters k) (or (get ind i) (op)) ind)))
        (put res i (vec op ;call-buffer)))
      res)
    (op ;inds)))

Some usages:

> (vec + [1 2] 3)
@[4 5]
> (vec + [1 2 3] [4 5 6])
@[5 7 9]
> (vec + [1 2] [3 4 5])
@[4 6 5]
> (vec + [1 2 3] [4 5])
@[5 7 3]
> (vec * 10 [1 2 3])
@[10 20 30]
> (vec + [[1 2] 3] 4 [5 [6 7 8] 9])
@[@[10 11] @[13 14 15] 13]
> (vec - [1 2 3 4] [1])
@[0 2 3 4] # NB. not @[0 -2 -3 -4]

Other usages on non-operators:

> (vec math/pow (range 10) 2)
@[0 1 4 9 16 25 36 49 64 81]
> (vec string/format "%c" (range 65 91))
@["A" "B" "C" "D" "E" "F" "G" "H" "I" "J" "K" "L" "M" "N" "O" "P" "Q" "R" "S" "T" "U" "V" "W" "X" "Y" "Z"]

You could even map a vectorization, if you wanted to:

> (map vec [+ - *] [1 [2 3 4] 5] [6 7 8])
@[7 @[-5 -4 -3] 40]

---

Addendum: I don't know of any other language that defines a vec function that operates in this way. Most languages either vectorize by default (Fortran, APL et al.), or have specialized types that override operators to vectorize (e.g. numpy). The only other language I know of that implements it separately is Julia, which refers to it as "broadcasting."

[sogaiu]

The last example:

(vec [+ - *] [1 2 3] [4 5 6])
# ->
@[5 -3 18]

reminds me a bit of juxt / juxt*:

(juxt* & funs)

Returns the juxtaposition of functions. In other words, ((juxt* a b c) x) evaluates to [(a x) (b x) (c x)].

[primo-ppcg]

Hah!

(map vec [+ - *] [1 2 3] [4 5 6])
# ->
@[5 -3 18]

If you actually wanted to do that, well there it is :laughing:

[nstgc]

If this works like it seems to, it would be like how in Julia the . postfix operator modifies the function dispatch (or something) to vectorize the function. Which is extremely handy.

edit: I didn't read carefully enough: you mention Julia. :sweat_smile:

[primo-ppcg]

I also think it's useful. Although perhaps defining it as operator modifier rather than a mapping function would be more clear (more akin to how Julia defines it, anyway):

> ((vec *) 10 [1 2 3])
@[10 20 30]

Anyway, after 3 weeks with very little feedback, and recent conments about removing rarely used functions (juxt and juxt* are the examples given), I think it's safe to say that it's not a feature that people have been waiting for.

[nstgc]

I also think it's useful. Although perhaps defining it as operator modifier rather than a mapping function would be more clear (more akin to how Julia defines it, anyway):

> ((vec *) 10 [1 2 3])
@[10 20 30]

Anyway, after 3 weeks with very little feedback, and recent conments about removing rarely used functions (juxt and juxt* are the examples given), I think it's safe to say that it's not a feature that people have been waiting for.

Yeah, ((vec x) body) does seem to make more sense. But yeah, if people are looking to cut something like juxt (which you might say I use a fair bit considering I use it at all while not touching half of the library), then vec seems DoA.

To make it an operator, you'd just wrap the body of the function you posted in a fn, right?

I think part of the issue is Janet versus Julia's use cases. Though Janet is generally much faster than Python (by a factor of three in my tests), there exist languages like Julia, R, and Matlab. There are plenty of reasons not to use the last two, but Julia is free and quite good, though I personally find that while it makes writing code easy, it makes writing good code harder (this is my personal experience as someone with a background in mathematical physics who has zero formal CS training) . Anyway, the point is that vec seems mostly suitable for numerical work and there are just better languages for that. Clojure, for example, is pretty decent at numerical work and it's very similar to Janet. It has a slow start up time, but that doesn't matter when dealing with long running applications.

For what it's worth, earlier this year I decided to write, from scratch, a basic neural network program. I did so in four langauges: Haskell, Julia, Clojure, and Janet. I never got the Haskell version to work; though the Julia version was the first specifically because of it's ability to "broadcast" arithmetic operators, the code quickly devolved into speghetti; the Clojure and Janet code were almost identical and both worked, but the Janet version exhibited severe numerical instability and was much slower at that. Elaborating based on memory (which with only one cup of tea in me isn't great this early in the morning), the Janet version converged about 10 times slower and often wouldn't converge at all unless the starting network was already close to optimized.

If I ever revisit that, the vec operator will definitely be useful. I have a personal library of useful functions, and vec is definitely one of them! Thanks!

[primo-ppcg]

Yeah, ((vec x) body) does seem to make more sense. But yeah, if people are looking to cut something like juxt (which you might say I use a fair bit considering I use it at all while not touching half of the library), then vec seems DoA.

In fairness, juxt* could probably go, and comp and partial could be macros, for the sake of speed. comp in particular could be written much more simply:

(defmacro comp
  [& functions]
  (with-syms [n res]
    (var n (length functions))
    (var res ~;xs)
    (while (>= (-- n) 0)
      (set res ~(,(in functions n) ,res)))
    ~(fn [& xs] ,res)))

To make it an operator, you'd just wrap the body of the function you posted in a fn, right?

Essentially.

(defn vec
  ``Vectorizes `op`. The resulting function will apply scalars to each item of indexed types, and
  indexed types pair-wise, with the shorter being padded with the operator identity.``
  [op]
  (fn f [& inds]
    (var len 0)
    (def ninds (length inds))
    (def iters (array/new-filled ninds))
    (loop [[k ind] :pairs inds :when (indexed? ind)]
      (set len (max len (length ind)))
      (put iters k true))
    (if (> len 0)
      (do
        (def res (array/new-filled len))
        (def call-buffer (array/new-filled ninds))
        (forv i 0 len
          (eachp [k ind] inds
            (put call-buffer k (if (in iters k) (or (get ind i) (op)) ind)))
          (put res i (f ;call-buffer)))
        res)
      (op ;inds))))

((vec +) [[1 2] 3] 4 [5 [6 7 8] 9])
# ->
@[@[10 11] @[13 14 15] 13]

I agree with most of your assessments. I've also found Janet to be much faster than Python, and I've also found Julia to a nightmare to maintain for anything other than toy scripts. When the Julia devs, in their wisdom, decided that while loops should be locally scoped - in other words functionally useless without a global keyword - is when I gave up on it. I've never used Clojure, although I've seen enough to know that it's ugly compared to Janet. And as for Haskell... there have been many attempts to make a Haskell-like language that doesn't suck. All have failed, as far as I'm aware :wink:

I also agree that Janet doesn't seem to be built specifically for numerical/mathematical purposes, although that is how I've been using it. But the language is quite expressive, so that most "missing" functional forms can be written simply and efficiently in just a few lines, as above.

[nstgc]

I also agree that Janet doesn't seem to be built specifically for numerical/mathematical purposes, although that is how I've been using it. But the language is quite expressive, so that most "missing" functional forms can be written simply and efficiently in just a few lines, as above.

Interesting. I'm too embarrassed to share my code, but might you have any insight into why the Janet code might be unstable and slow to converge? At first I thought Janet was using 32b floats and Clojure 64b, but a quick check in the docs shows this isn't the case. The Janet version was written first due to ease of debugging, and then the Clojure version, so if it were a case of transcoding error, it would have to have been a beneficial one. Even then, the Julia code lacks this issue as well, and as mentioned, it was the original implementation.

Also, if you don't mind my asking, what do you use Janet for? For me, I mostly use it for simple system scripts and string processing. Most recently, I've been working on a script that converts between markup languages. I started largely because my D&D group is moving from using Google Docs for lore and homebrews to a Doku Wiki I'm hosting. Pandoc converts Google Docs to HTML and then the Janet script converts that to an IR and then Doku format. I also started on a LaTeX->IR module with the intention of using it in place of Pandoc, which really doesn't work well for me.

I've never used Clojure, although I've seen enough to know that it's ugly compared to Janet.

As for this, to each their own of course, but I think it's mostly Java baggage. I use Clojure when there's a relevant library. Which is very often the case given the breadth of the Java ecosystem.

Edit: Oh, and thank you got the updated vec!

[primo-ppcg]

Interesting. I'm too embarrassed to share my code, but might you have any insight into why the Janet code might be unstable and slow to converge?

Perhaps you could open a discussion about it. I wouldn't worry to much about the "quality" of the code, I'm sure we've all written worse :wink:

Also, if you don't mind my asking, what do you use Janet for?

Coincidentally, also mostly for data transformations, of the numeric kind. In particular, I've been using it for generating parametric data from radial coordinates which follow a specific spheric curvature, such that the resulting edge along the substrate has a specific curvature (generally not the same as the substrate itself) and a specific total length. There's a number of parameters which affect the calculation, and a number of output formats depending on the machine it's being fed to. We have a relatively large corpus of Python code that I've been slowly converting to Janet, all of which has become shorter and faster.

Of course, I also use it in my free time for pointless mathematical inquiries :smile: