search

tc39 / proposal-pipeline-operator

A proposal for adding a useful pipe operator to JavaScript.
http://tc39.github.io/proposal-pipeline-operator/
BSD 3-Clause "New" or "Revised" License
7.53k stars 108 forks source link

Consider Elixir-style pipeline as one of the blessed alternatives #143

Closed littledan closed 3 years ago

littledan commented 5 years ago

I know we previously discussed this at length and settled on F#-style, but I keep meeting developers who have the intuition that pipeline should be Elixir-style, inserting the argument at the beginning. Would anyone be interested in writing this up formally and implementing it in Babel, so we could get hands-on experience with this alternative?

mAAdhaTTah commented 5 years ago

Related to #20.

I suspect whomever wanted to implement this in babel could piggy-back off the F# parsing and just write an additional babel transform.

littledan commented 5 years ago

Right, I see that there were reasons for not going with #20, but it keeps coming back.

Piggy-backing off of F# for the implementation sounds good. I think it'd also be helpful to have an explainer document and maybe a draft specification to document it and make it more concrete. There are some edge cases to think through, e.g., should it be inserted into the argument list if there are parentheses around the thing on the right hand side of the |>.

If anyone wants to take this on, please say so here and I'm happy to help you get started.

js-choi commented 5 years ago

I’d be happy to draft an explainer and specification for a tacit-first-argument-only pipe operator later, although for now I’ll be currently working on the Babel transform for the smart-pipeline proposal.

We also will need a name for the new proposal (see #128). I suppose Elixir style might do...

littledan commented 5 years ago

Let's go with Elixir for the name--we've been using it in this repo, and there's a lot of overlap between Elixir and JS programmers, so it could be a useful reference.

zenparsing commented 5 years ago

There are some edge cases to think through, e.g., should it be inserted into the argument list if there are parentheses around the thing on the right hand side of the |>.

This is going to be the troublesome part, and it's called out specifically in the Elixer docs.

We might also want to consider whether the old bind proposal can be updated to insert the LHS as the first argument instead of the this value.

js-choi commented 5 years ago

Indeed, accommodating both Elixir’s first-argument style in addition to last-argument styles without too much magic is one of the goals of the current smart-pipe proposal—although, of course, that other proposal currently makes other trade offs in exchange for such an advantage.

In regard to “I keep meeting developers who have the intuition that pipeline should be Elixir-style, inserting the argument at the beginning”, it’d be good and helpful to hear from some of those developers or to see some concrete examples of the code they specifically have in mind.

littledan commented 5 years ago

My impression so far is pretty simple, roughly: "this is all solved in [xyz language] and it's not so complex; I don't understand what you're worried about". It would be good to discuss more, though, I agree.

gilbert commented 5 years ago

To summarize the biggest downside to Elixir-style: function programming enthusiasts were concerned that they would not be able to use curried functions within a pipeline. For example:

var add = x => y => x + y;

var result = 10 |> add(20);

// Elixir-style would translate the above to:
var result = add(10,20);
// which does not work with the way `add` is written.

Someone proposed (I forget who) a syntactic solution using parenthesis:

var result = 10 |> (add(20)); //=> 30

But many were not happy with it. Personally I think it's a good compromise, and even makes sense: the standard behavior of parethesis is to evaluate the inside before the outside.

js-choi commented 5 years ago

If I’m understanding this correctly, then, the Elixir style for which these people have been asking would interpret function/method-call syntax as a special case: x |> f(a, b) would be f(x, a, b), x |> o.m(a, b) would be o.m(x, a, b), and x |> anyOtherTypeOfExpression would be anyOtherTypeOfExpression(x). If I’m understanding this right.

But if I’m understanding it correctly, then that runs into ambiguity problems—the same ones that made me propose restricting smart pipelines’ tacit syntax and forcing everything else to be explicit with the placeholder. For instance, with the Elixir pipe, what would x |> {}.m(a, b) mean? What would x |> f(a)(b) mean?

littledan commented 5 years ago

OK, thanks for this summary, I will try to circle back with advocates of this variant and see their thoughts.

bjunc commented 5 years ago

As an Elixir developer, it was my assumption that the style was going to be "argument first" (as described by @gilbert and @js-choi above). Per the question above, I think it would look this this, no?

// x |> f(a)(b)
function f(x, a){
  function(b) {
    ...
  }
}

Personally, my favorite part of pipelines are their simplicity, and would be fine if "advanced" syntax just wasn't supported as part of the pipeline (you can always put it into one of the functions). In-fact, IMO, I see these advanced syntax situations as almost oxymoron to using pipelines.

Looking forward to the day that I get to write this in JS:

viewer =
  request
  |> accepts(['json'])
  |> verify_header(claims, { realm: 'Bearer' })
  |> ensure_authenticated()
  |> verify_not_blacklisted()
  |> load_resource()
mAAdhaTTah commented 5 years ago

I find the Elixir-style pipe un-JavaScript-y. I can't speak to the intuitions of those who expect that, but I feel like it would be confusing for most JavaScript devs.

I'd also be a little concerned about making the communication around the operator more confusing overall if we add another proposal to the mix. I wouldn't want to add this to the mix unless we saw a strong push for this behavior.

littledan commented 5 years ago

Well, informally, I hear a lot of negativity about smart pipeline, and a lot of people asking for Elixir, but this isn't very scientific.

littledan commented 5 years ago

To clarify, I think we should consider things very open and unsettled at this point, and consider all three options, if we can manage to come up with some kind of initial guess for what to do about the parentheses semantics.

dead-claudia commented 5 years ago

I would like to mention just for historical and informative purposes this is closely related to some past discussion (1, 2, 3, 4, 5) in the bind proposal repo. It's also where this grew out of, and I feel it's very much so worth looking through those to pick apart the foundations of this, so we don't forget what led us here and so we don't lose track of the original intent of the proposal. (I feel we've gotten a little too heavily invested in how it interops with mostly pure functional code, at the cost of how it works in the common case of simple pipelining and how it works with procedural code.)

robgraeber commented 5 years ago

I agree with @mAAdhaTTah that built in currying is unintuitive for a native javascript operator. Are there any other cases of javascript supporting currying natively?

zenparsing commented 5 years ago

A possiblity that combines Elixer-style first-arg passing with the syntactic structure of the bind proposal (as suggested above by @isiahmeadows).

Syntax:

CallExpression:
  CoverCallExpressionAndAsyncArrowHead `->` MemberExpression Arguments
  CallExpression `->` MemberExpression Arguments

Common usage scenario:

import { map, filter, collect } from 'iter-funs';
someIterator->map(cb1)->filter(cb2)->collect().forEach(cb3);

Pros:

Cons:

Thoughts?

charmander commented 5 years ago
resolved > Developers don't have to write a specific version of functions to match the pipeline style (e.g. underscore just works). They do, it just happens to be what’s used by e.g. underscore. For something existing that won’t match, see current RxJS.
zenparsing commented 5 years ago

Let me clarify: first-arg style is well supported by the language outside of any pipeline feature (be it syntax or function). If I create a library I can just choose that style and I know it will work well regardless of whether my users are using pipeline-things or not.

Current RxJS, on the other hand, requires use of a "pipe" method (or pipeline operator) to make things ergonomic.

dead-claudia commented 5 years ago

@zenparsing

  • Usage of -> might be surprising for those with C++ experience.

I wouldn't consider this a significant con considering how few JS devs use C++. You could make a better argument for PHP, since their -> is effectively our ..

I would posit the second con, about native support for point-free support, is probably more significant, but I'm not convinced it really carries the benefits the FP community say it does in JS. JS FP works more like Elixir and Clojure than OCaml or F# - arity is a concern, and it's impractical to define any generic auto-currying mechanism because callees can have multiple arities.

As for what primitives they have for flipping function application:

I did note when I first proposed it that one of the other major pros (one you didn't list) is that it's already a very prevalent idiom. Lodash, Underscore, and jQuery's array/object utilities all use the first argument for the data itself, so users could start using it right away and we wouldn't need to tell everyone to change all their idioms.

dead-claudia commented 5 years ago

And another pro with a->b(...xs) ↔ b(a, ...xs): a->await b(...xs) is pretty obviously not ambiguous, and it obviously evaluates to await b(a, ...xs). It dodges the whole issue of how to handle async and yield by just not letting the RHS be just any expression.

dead-claudia commented 5 years ago

Also, just wanted to mention to others here that @zenparsing's idea was something I first suggested here a while back, coming from the context of iterables and generators.

littledan commented 5 years ago

For more context, -> has been previously discussed for a short function literal that doesn't bind this, but I am not convinced we should add a feature for that (given how confusing this is already).

I'm not sure the property access confusion concern is C++-specific, as @zenparsing has specifically advocated a pattern like this for private state.

gilbert commented 5 years ago

@isiahmeadows makes a good point about curried vs uncurried languages. I agree that arg-first is already idomatic JavaScript.

To add another case study, ReasonML has chosen -> for their pipeline function. This is especially interesting considering the operator syntax was originally |>.

mrsufgi commented 5 years ago

didn't quite understand why -> works and |> don't. ״We also cannot use the |> operator here, since the object comes first in the binding. But -> works!״

dead-claudia commented 5 years ago

@mrsufgi Reason actually has both F#-style and Elixir-style forms, but the docs omit this bit:

You'll see a lot of older code using the first, since the second is relatively new.

But it's especially notable that Reason specifically added x->f(g) ↔ f(x, g) for both JS and native targets, despite x |> f(g) ↔ f(g, x) already existing as a viable, working alternative.

mAAdhaTTah commented 5 years ago

@js-choi To be honest, I'm not terribly keen on a "Split Mix"-style operator, as I find similar operators with different semantics too complicated to be worthwhile. I'm not opposed to doing Elixir-style pipelining with a different operator (e.g. ->), although I don't know how the committee would feel about 2 (or potentially 3, if we include bind) operators for various forms of pipelining.

littledan commented 5 years ago

I'm skeptical of having multiple pipeline-related operators. We should have a pretty good understanding of why we really need both. Less to learn is better, all else being equal. But interesting to hear that other languages went that way.

rbuckton commented 5 years ago

I'm concerned introducing or using an Elixir-style operator would mean TC39 blessing first-arg semantics which would have repercussions for existing userland libraries like Ramda that use last-arg semantics. That was one of the motivators for partial application, in that you could place ? in either first-arg or last-arg positions.

zenparsing commented 5 years ago

It's a good point @rbuckton. Favoring a certain style is unavoidable unless we go with something like hack-style/smart-mix or couple pipelining with partial application.

We do have to consider that coupling proposals multiplies risk, though.

js-choi commented 5 years ago

For future readers: If @mAAdhaTTah’s https://github.com/tc39/proposal-pipeline-operator/issues/143#issuecomment-458715646 and @littledan’s https://github.com/tc39/proposal-pipeline-operator/issues/143#issuecomment-458769275 are confusing, it’s because I deleted a comment in which I mused about someone reviving the old “split-mix” system in a formal proposal, in which different pipe styles (Elixir, F#, Hack) are used by different operators, which would favor no particular style. I deleted it because I wanted to think about it more before anyone saw it but it looks like it was too late, heh. It seems to probably be a nonstarter anyway.

I do think that not favoring a particular system is an important goal. It’s a big part of why I’m enthusiastic about any proposal that can comfortably accommodate multiple/mixed styles (whether “smart” or “split” or whatever). I think there are still unexplored ways of reconciling different mixed styles, including mixed styles supporting Elixir style.

chenglou commented 5 years ago

I maintain Reason, mentioned by @gilbert. Someone asked me to justify it here, so here I am.

There are very nuanced but important distinctions to have pipe first rather than last, that warranted going the extra length of adding it rather than using the existing |>.

The more important issue is outlined e.g. here. Type inference, and general editor tooling, don't work well at all with last-arg, akin to the issue of import foo from bar rather than from bar import foo: you need some epic analysis to automplete foo in the former case. The truth is that all the tools are top-down left-right biased. For example, map(a => addInts(a, b), myStringArray) will give incorrect hint (from the compiler, editor tooling including autocompletion and type hint, refactoring, etc) that myStringArray has the wrong type. It's usually the case that the function is the wrong type (data over functions, functions over macros, etc). map(myStringArray, a => addInts(a, b)) will correctly warn at the more local addInt part.

FP literature kinda traditionally ignored these ergonomic problems, because "as long as the compiler errors it's theoretically fine". For a real-world, user-friendly production language, the quality of the compiler's help matter a lot. You start using different designs at that point.

To generalize this, if we check the FP languages themselves, especially those with currying, we'd realize that arg-last isn't well-defined (in the most serious sense). Consider:

int => float => string => int

It looks like string is the last argument (last int is the return type). But look:

type t = string => int 
type myFunc : int => float => t

Now it looks like float is the last argument! It's not just a matter of looks. These languages usually can't tell the difference. Consider if t was polymorphic and you couldn't just inline the types or something. So, in a curried language, you can only know which one is the first argument, but can not tell which one is the last argument. The wrecking of editor tooling earlier is one of the many manifestations of this.

The history of arg-last in JS libraries are to emulate actual FP languages' semantics. In turn, the semantics of arg-last in those languages are usually undefined in the context of arg-last pipe. Imo it'll be legacy decision in the making, to go with arg-last pipe for JS.

Lastly, the awkward and unspoken argument for arg-last comes from the fact that some find neat little elegance in the pipe in conjunction with automatic partial application. That's not a thing in JS (clarification: currying can be, but we're taking about auto partial application, when you call the function). And if we're even on the topic "neat parallels", OO methods being just a regular function that takes an object as first argument, seems like a more "elegant" analogy for FPs to make, rather than some other neat-parallel-with-currying-but-not-really-that-elegant analogies anyway.

Honestly, take it from the creator of TypeScript from the link I provided above. That person seems to know what he's doing =).

tbremer commented 5 years ago

@chenglou thanks for the thorough response. I have always wondered why Bucklescript supported both operators.

dead-claudia commented 5 years ago

I'd like to bump the original request of adding the Elixir-style semantics of x |> f() ↔ f(x) + x |> f(...) ↔ f(x, ...) to the list of considered alternatives, just to match reality. I've seen it talked about in the notes as well as extensively in the issues, but the README and wiki don't seem to match this reality.

hax commented 4 years ago

I feel Elixir-style pipeline is like the compromise of OO world and FP world, though both world may accept it, both world may also not satisfy with it...

fabiosantoscode commented 4 years ago

The elixir pipeline has a very interesting feature in that you don't have to curry your non-unary functions for them to work with it.

Indeed it can help replace the current pervasive "chaining" strategies. Currying is cool, but it's not widely used in JavaScript. The "object" of the function being the first argument is.

The current proposals which I've been able to find (unary functions only, if you need more arguments you need an inline arrow function or to curry your function) will only cause more boilerplate and confusion. Does this package curry its functions? Did I curry this function? Let me curry this function so people can use pipelines with it.

The simple syntax x |> foo() being equivalent to foo(x) has a tiny learning curve, doesn't affect anything negatively and has the potential to be a great boon to code reuse.

Edit: forgot to mention that clojure's similar -> macro works the same way. Check it out: https://clojure.org/guides/threading_macros

So two very functional languages don't force you to curry or change the signature that makes the most sense for regular usage when you just want to pipe several functions. What then is the reasoning for this requirement in this proposal?

fabiosantoscode commented 4 years ago

I've read someone concerned with performance. Since this proposal is syntax sugar, your minifier will take care of desugaring it, such that it has less characters and parses faster. Terser will support this for sure, that's a Promise I can certainly keep.

mAAdhaTTah commented 4 years ago

@fabiosantoscode The performance concerns are typically about the overhead of creating curried functions, not size.

fabiosantoscode commented 4 years ago

Got it! Yes, that makes sense. Currying is not free.

chenglou commented 4 years ago

Lemme clarify that for our pipe-first implementation, we just do a syntax transform back to the normal function call. Doesn't disrupt any existing semantics, extremely predictable and toolable, and no runtime performance hit, obviously.

highmountaintea commented 3 years ago

First argument topic is how I envision Javascript would be in the FP world. It has worked very well for Elixir. It gels well with how function arguments are handled in JS, including variadic arguments. It's extremely consistent with how vanilla JS functions work. It is also performant compared to alternatives.

Using String.split() as an example: the functional equivalent of String.split([separator, [, limit]]) would be split(topicStr, [separator, [, limit]]). It's very intuitive, and does not require a function that returns another function or such. Same with String.substr(), String.indexOf(). But the same would apply to other libraries in general, like base64encode(), JSON.parse(), sha256hash() etc.

Let's say I want to write a Date library, would I want my function to be of the form addDays(date, numDays) => newDate, or would I want it to be of the form addDays(numDays)(date) -> newDate? I feel first argument topic is a more natural way of doing FP in Javascript.

tabatkins commented 3 years ago

First-argument injection is, I feel, the least good of all the options presented so far.

highmountaintea commented 3 years ago

@tabatkins thank you for your comment. Those are all valid concerns. I wrote down some of my thoughts regarding your comment, but decided to hold it off until I come up with something a little more concrete.

highmountaintea commented 3 years ago

First-argument injection is, I feel, the least good of all the options presented so far.

  • Your code doesn't read like it was defined - all the arguments in a pipeline call are in the wrong position relative to calling the function literally.

I agree. My reply is this is probably a case where the concern is not born out in practice. Clojure, Elixir, Racket all implement argument injection as their pipe mechanism, and users of those languages do not find it confusing.

  • It's an entirely novel calling syntax for JS. Both F#-style and Hack-style leverage existing JS syntax/calling idioms. (This isn't necessarily a strike against, but it means there has to be a very good reason why introducing such a novel syntax is worthwhile, vs just reusing existing familiar syntaxes.)

I presume you meant it is a novel mechanism (instead of syntax)? I would say this mechanism is very easy to get used to, as born out in practice in all languages that implement it.

  • Most importantly imo, if your function is not written in a way such that you want to pass the topic into the first argument, adjusting your pipeline body to account for that is very non-obvious and strange. As far as I can tell, you have to write something like val |> (x=>foo(1, x))()? Maybe you can omit the final parens if there's a rule that when the top-level expression isn't a function call it acts like F#-style and implicitly calls it with the topic as the sole argument, allowing val |> x=>foo(1,x)? (Assuming, as with F#-style, that we satisfactorily solve the parsing issues with omitting wrapping parens around arrow funcs in pipe bodies.)

Yes, I agree. That's where Hack proposal would be useful :) My main gist is that argument injection is not a novel idea, and works well in the languages that already implement it. These languages typically pick argument injection as their default pipe mechanism, and argument replacement as the fallback. We shouldn't reject it out right, and it may have some synergy with the Hack proposal.

I decided to write a new proposal that combines both argument replacement and argument injection to illustrate my points. Maybe it can alleviate some of the concerns: https://github.com/tc39/proposal-pipeline-operator/issues/84#issuecomment-831328201

lightmare commented 3 years ago 
* Your code doesn't read like it was defined - all the arguments in a pipeline call are in the wrong position relative to calling the function literally.

I'm going to agree with @highmountaintea that this doesn't tend to be an issue when the syntax is clear and unambiguous. This function—argument order inversion can even be seen in non-pipe contexts, for example method calls in Python:


class K:
# definition order is: funcName, subject(self), *args(x, y)
def funcName(self, x, y): pass

obj = K()

call order is: subject, funcName, *args

obj.funcName(x, y)

And it doesn't appear to cause trouble once you get used to it.

>     * It's an entirely novel calling syntax for JS.

For the language, yes, but there is precedent in libraries. For example Lodash:
```js
// the basic form of lodash functions operate on the first argument:
_.mapValues( _.groupBy(bills, 'type'), g => _.sumBy(g, 'amount') )

// but you can also wrap your input in a proxy and pipe it like this (and since we're talking
// about syntax, let's ignore the semantic difference that this evaluates lazily; what's notable
// is that these methods are not documented individually, they simply defer to the documentation
// for the functions operating on their first argument):
_(bills)
.groupBy('type')
.mapValues(g => _(g).sumBy('amount'))
.value()

// which is very similar to how you would use the regular no-proxy functions with Elixir-style pipeline:
bills
|> _.groupBy('type')
|> _.mapValues(g => g |> _.sumBy('amount'))

First-argument injection is, I feel, the least good of all the options presented so far.

To me it feels like the most balanced of all the alternatives presented.

Now why do I think the Elixir-style strikes a good balance between the two? Because F#-style is only really good with unary functions. Elixir-style is good whenever you want to pipe the first argument, i.e. unary functions and then some, and the price you pay over F# for that wider range of easily usable functions is mere ().

Elixir-style balance

With unary functions - F# wins:

input |> func // F#
input |> func() // Elixir
input |> func(#) // Hack
func(input) // desugar

Binary with topic first - Elixir wins:

input |> (x => func(x, 2)) // F#
input |> func(2) // Elixir
input |> func(#, 2) // Hack
func(input, 2) // desugar

Binary with topic second - Hack wins:

input |> (x => func(1, x)) // F#
input |> (x => func(1, x)) // Elixir **1 (implicit arrow-function call)
input |> (x => func(1, x))() // Elixir (explicit call)
input |> func(1, #) // Hack
func(1, input) // desugar

**1 this is the extension proposed earlier. Elixir-style requires a call-expression on the right-hand-side. If there's an arrow-function-expression instead, it's safe to assume you want it called. Of course you can add explicit (), but they're redundant, the function will be called either way. As for removing the parentheses surrounding the arrow function, that's a can of worms affecting any style that allows it, so no point lifting the lid here.

Mid-pipeline await:

input |> asyfunc |> await |> regfunc // F#
input |> await asyfunc() |> regfunc() // Elixir **2
input |> await asyfunc(#) |> regfunc(#) // Hack
regfunc(await asyfunc(input)) // desugar

**2 this is another extension. Again, Elixir-style requires a call-expression on the right-hand-side. If there's an await-prefixed call instead, you want to await the result (similarly with yield).

Note that unlike special parsing rules needed for await in F#-style, in Hack-style |> is a regular binary operator, and in Elixir-style it's almost like a regular binary operator, only with right-hand-side restricted to certain expression types (call or arrow, optionally prefixed with await or yield; i.e. existing productions, no new restrictions on specific token sequences).

ducaale commented 3 years ago

For completeness sake, this is what minimal pipelines + partial application proposal + some helpers would look like:

input |> func
input |> func(?, 2)
input |> func(1, ?)

const Promise = pipeable(globalThis.Promise) // https://gist.github.com/ducaale/93bd6d49314ef1383f50be95edca9d6e
await (input |> asyfunc |> Promise.then(regfunc) |> Promise.then(regfunc2))
rbuckton commented 3 years ago

For completeness sake, this is what minimal pipelines + partial application proposal + some helpers would look like:

input |> func
input |> func(?, 2)
input |> func(1, ?)

That was part of the initial rationale for partial application, to allow both leading-arg (lodash style) and trailing-arg (Ramda style) usages (along with its other capabilities).

For me, both F#-style (w/partial application) and Hack-style feel like JavaScript, with semantics that fit with the language. Elixir-style doesn't quite fit with JavaScript semantics, and I'm concerned it would introduce a footgun that is the polar opposite of the confusion around this (where instead of a hidden parameter, we now have a hidden argument).

lightmare commented 3 years ago

I wrote a draft patch for the Babel plugin. You can try it in this experimental REPL (requires manually setting Presets -> Pipeline proposal -> Elixir)

lightmare commented 3 years ago

Another Elixir-style pipeline and capture operator playground I've been experimenting with: engine262 fork

tabatkins commented 3 years ago

Closing, since the proposal has advanced with "Hack-style" pipeline syntax.