Improve partial application

[sirlensalot]

Currently partial application is implemented as follows:

1. Natives magically implement as one-offs, using Pact.Eval apply,apply'
2. the typechecker relies on the type signature of the consuming function to detect partial application, using that to mangle the type signature to add the provided arguments. Basically a function type in argument position assumes partial application.
3. flip is impossible currently which then requires new functions for flipped versions of >, etc.

One approach is to introduce wildcards a la Clojure, underscores, or maybe question marks are better as they at least resemble SQL statement parameters: (> ? 20). Implementation would be relatively straightforward by making the arglist recognize the special wildcard, presumably with another constructor in the Arg type.

Thunks are a more robust solution, but with one big problem: there is no way to distinguish partial application from full application currently in Pact, the specific problem being overloads with different arities. The current Haskell-style partial-application is quite convenient to write, and would need partial or other legibility-damaging sugar for this. However, we could do what the typechecker does and just assume a function type in arg position is partial, and then require explicit wildcards when wanting to do a thunk in some other position.

Thunks would either be a new AST member or a flag in TApp Term constructor.

[pdillinger]

This area of the language is rare enough not to warrant significant extra complexity just for some code brevity. Brevity, particularly in rare contexts, can of course be the enemy of readability and implementation correctness, particularly if you want the language to be Lisp-like and the enhancements (including existing partial application, compose, etc.) depart from Lisp, or its simpler-and-not-broken nephew Scheme. (In particular, Scheme got rid of the utterly confusing #\' operator in Lisp.)

I think that defined function symbols and lambda literals should be permitted in "immediate application" contexts, which in Pact would include the functional argument of map, fold, and filter. Consider a Scheme example (tested in Racket):

> (map length '(() (1 2 3)))
'(0 3)
> (map (lambda (x) (+ 1 (length x))) '(() (1 2 3)))
'(1 4)

That's super easy to read, easy to process, and super flexible/powerful. It's just somewhat verbose, and that's not worth leaps and bounds to fix. Just shortening the name of "lambda" would go a long way.

To emphasize the commonness of lambda, even ACL2, a Lisp-based system that is first order (no functions as values), recognizes immediately applied lambdas, to support let and let* as macro expansions:

ACL2 !>((lambda (x) (+ 1 (length x))) '(1 2 3))
4

[sirlensalot]

Partial application is already important and widely used in pact, not a "rare area", employing a Haskell-like look that is drastically more preferable from a legibility point of view than partial and other LISP partial app stuff I've seen. Here's an example of how it gets used in select:

(select people (and? (where 'age (> 30)) (where 'name (= "Fatima"))))

The point is taken that a compressed syntax like wildcards could be dangerous. Regarding "immediate application" lambdas, a general concern with lambdas in Pact is the possibility of forming a Y-combinator, does this "immediate application" address that?

In many respects the biggest problem right now is the UX surrounding comparators: in the example above, it's easy to think the where is selecting users over 30 when instead it's "30 is greater than". One option is to introduce flipped versions of comparators, perhaps using the name mangling employed above on and? (a higher-order and for composing partial applications), ie (>? 30).

[pdillinger]

a general concern with lambdas in Pact is the possibility of forming a Y-combinator, does this "immediate application" address that?

Yes it does. You have to allow binding symbols dynamically to functions/closures to build a combinator. For an "immediate application" context we must know that the lambda/closure is never bound to a symbol exposed to the user, nor that it "escapes" by return from a function or passage to a user function.

To your whitepaper point, "but also avoids creating closures which can significantly complicate variable scoping, leading to bugs," immediate application means that the dynamic vs. lexical scoping distinction never comes into play; immediate application means that the dynamic and lexical scopes are the same.

I certainly concede that lambdas do not solve the problem of concisely binding table columns to canonical names. One way to do this could be to have a syntactic distinction (sigil) for implicitly-bound names, and 'where' (or 'where$') would be a lambda-like built-in that binds such names, e.g.

(select people (where$ (and (> $age 30) (= $name "Fatima"))))

(Resolution of such names might be dynamic for REPL and during typecheck for analysis--I haven't fully thought it through.)

[fosskers]

I hit the same confusion as @slpopejoy during some recent experimentation. Consider:

-- Haskell
filter (< 5) [1..10]  -- [1,2,3,4]

and

;; Pact
(filter (< 5) [1 2 3 4 5 6 7 8 9 10])  ;; [6 7 8 9 10]

I'd advocate for a Clojure/Scala-like wildcard system for lambdas, something like:

;; Pact
(filter (< 5 _) [1 2 3 4 5 6 7 8 9 10])  ;; [6 7 8 9 10]