Feature: Functions as first class objects

[CasparKielwein]

Feature Request

Add a possibility to pass functions as parameters and store them in variables and collections. This allows the use of abstract algorithms like map, filter, reduce,...

This feature request is not about anonymous functions aka closures / lambdas.

The natural language like style, where parameters are interleaved in the function signature and call makes implementation of this feature more difficult than in other languages.

Possible limited solutions and workarounds:

• Only allow unary functions with signature "name {parameter}".
• A string eval function

Use Cases

Allow users to write abstract algorithms which depend on behavior injected by the caller. Example: sorting a collection of collections by an arbitrary element.

I am working on a proof of concept to include Jinx as a command and scripting language in lighting control software. I would like to enable users to write their own transformations and filter functions to select objects by arbitrary criteria.

Examples in other languages

• C allows passing of function pointers as parameters. In particular the qsort sorting function uses this feature.
• lua stores functions in tables.
• C++ has multiple such features including the generic std::function wrapper.
• Functional programming languages are essentially built on this concept.

Related features: #11

[JamesBoer]

I've actually given this feature some thought based on coroutine design work. It probably makes sense to roll this out in two stages, since first-class functions the logical precursor to coroutines anyhow. Initial design may look like this:

-- Function declaration
function multiply {x} by {y}
    return x * y
end

-- Assign a specific function to a variable 'f'
set f to function multiply {x} by {y}

-- Execute specified function by variable
set x to call f 3, 4

Here's the same example combining function declaration and variable assignment in one step for convenience.

-- Function declaration and assignment
set f to function multiply {x} by {y}
    return x * y
end

-- Execute specified function by variable
set x to call f 3, 4

I hadn't initially implemented first-class functions mostly because I hadn't really thought of a good way to reference or call generic functions, but I later realized I'm already doing that via the C++ API's Script::CallFunction(). So I think we can do the same thing in the language with a generic callfunction, and just pass all arguments as a collection.

I can't promise anything initially until I at least take a stab at implementing this and see if I encounter any serious issues.

[JamesBoer]

There's a very early preview of this feature in branch feature-fcfunctions. Documentation has been updated in that branch, so I'll let that speak for itself.

-- Required for call function
import core

-- Function definition
function multiply {x} by {y}
    return x * y
end

-- Assign a function to a variable 'f'
set f to function multiply {} by {}

-- Execute specified function by variable
set x to call f with 3, 4

Obviously, this still needs much more testing and feedback, but so far so good. I'm probably going to hold off with the shortcut method of combined function declaration and assignment for the time being.

[CasparKielwein]

Very cool, I'll have a look at it and play around with it a bit.

[CasparKielwein]

I implemented some higher order functions using the feature branch: https://github.com/CasparKielwein/jinx_playground/blob/main/functional.jinx
This implementation serves as a fairly nice proof of concept. I like how it works and found getting used to jinx quite easy.

I have one question / request: Why is function <signature> not parsed as an expression which returns a value of type function?

This is even more interesting when passing functions to other functions than directly assigning them to variables, The test of the higher order functions show this, where a separate variable and statement is always required before a function can be used in a higher order function.
https://github.com/CasparKielwein/jinx_playground/blob/main/functional_test.jinx

(I tried to implement that, but do not quite understand yet how jinx parses expressions.)

[JamesBoer]

I was always looking at the possibility of allowing definition / assignment shortcuts:

-- Combined function definition and assignment
set f to function multiply {x} by {y}
    return x * y
end

-- Execute specified function by variable
set x to call f 3, 4

I wanted to test out the core functionality before making those changes. I don't foresee this being too difficult to implement.

However, I wasn't really considering parsing them as expressions, but as a specialized form of the assignment statement. So it would be limited to this particular form. Syntax-wise, I'm having a hard time seeing how that would work as an expression in a more general case.

Let me know if you think the above example would be satisfactory, or whether you have some other design proposal or ideas.

[CasparKielwein]

Let me know if you think the above example would be satisfactory, or whether you have some other design proposal or ideas.

The proposal with the shortcut for assignment of function variables works for my use case.

My issue is more that the limit to assignment feels like a somewhat unnatural limitation. But allowing function declarations as expressions is a generalization of assignment and I see no issue in possibly adding it later without loosing backwards compatibility.

In informal terms I would like to do the following without having to assign the function variable before it:

• set a to 1, function foo {x}
  store a function in a collection
• call bar function foo {x}
  pass a function as a parameter

I will try to write up a more formal proposal which takes possible parsing and precedence issues into account.

[JamesBoer]

Actually, I was thinking about this the other day, and I belatedly realized I totally misunderstood. For some reason, I thought you were talking about parsing the function definition (signature + body) rather than just the function declaration (signature only) as an expression. However, parsing the signature as an expression makes perfect sense, and honestly, now I'm not sure why I didn't implement it like that in the first place. Anyhow, yeah, I'm definitely on board with your suggestion.

[JamesBoer]

This should now work as proposed on the feature-fcfunctions branch. Let me know if you run into any issues or have additional ideas.

[JamesBoer]

First-class function and coroutine support have been merged into development branch.

[CasparKielwein]

I finally got around to testing the feature again and it works really nice :+1:.

[JamesBoer]

Good to know, thanks!

[JamesBoer]

Feature has been merged into master branch.