Proposal: Hack-style Pipelining

[zenparsing]

The current proposal (in which the RHS is implicitly called with the result of the LHS) does not easily support the following features:

• Piping into a function with multiple arguments
• Calling methods on a piped value
• Awaiting a piped value

In order to better support these features the current proposal introduces special-case syntax and requires the profilgate use of single-argument arrow functions within the pipe.

This proposal modifies the semantics of the pipeline operator so that the RHS is not implicitly called. Instead, a constant lexical binding is created for the LHS and then supplied to the RHS. This is similar to the semantics of Hack's pipe operator.

Runtime Semantics

PipelineExpression : PipelineExpression |> LogicalORExpression

1. Let left be the result of evaluating PipelineExpression.
2. Let leftValue be ? GetValue(left).
3. Let oldEnv be the running execution context's LexicalEnvironment.
4. Let pipeEnv be NewDeclarativeEnvironment(oldEnv).
5. Let pipeEnvRec be pipeEnv's EnvironmentRecord.
6. Perform ! pipeEnvRec.CreateImmutableBinding("$", true).
7. Perform ! pipeEnvRec.InitializeBinding("$", leftValue);
8. Set the running execution context's LexicalEnvironment to pipeEnv.
9. Let right be the result of evaluating LogicalORExpression.
10. Set the running execution context's LexicalEnvironment to oldEnv.
11. Return right.

Example

anArray
  |> pickEveryN($, 2)
  |> $.filter(...)
  |> shuffle($)
  |> $.map(...)
  |> Promise.all($)
  |> await $
  |> $.forEach(console.log);

Advantages

• All use cases described above are supported with no special-case syntax.
• By removing the implicit-call behavior, we remove a potential source of confusion for developers.

Disadvantages

• By requiring an explicit function call, functional programming patterns that rely heavily on currying will be less ergonomic.

Notes

The choice of "$" for the lexical binding name is somewhat arbitrary: it could be any identifier. It should probably be one character and should ideally stand out from other variable names. For these reasons, "$" seems ideal. However, this might result in a conflict for users that want to combine both jQuery and the pipeline operator. Personally, I think it would be a good idea to discourage usage of "$" and "_" as variable names with global meanings. We have modules; we don't need jQuery to be "$" anymore!

[littledan]

Thanks for writing this out in such a detailed way. I didn't understand exactly what you were suggesting before. I like how this direction is very explicit and general; it completely handles nested subexpressions in a way that I didn't previously understand was possible.

The cost seems to be the loss of terse syntax for x |> f |> g, which becomes x |> f($) |> g($). I'm not sure how important conciseness for that case is.

Bikeshedding: I'm not sure if we want to use _ or $ when they are so widely used for libraries. There are lots of other names possible, or we could go with something currently untypable like <>.

[zenparsing]

I'm not sure if we want to use _ or $ when they are so widely used for libraries.

True, using $ or _ will probably make some people unhappy. On the other hand, we should prioritize the future over the past.

Another thought: a pipe operator with these semantics would obviate the need for the current :: proposal. We could re-purpose binary :: for just method extraction:

class XElem extends HTMLElement {
  _onClick() {
    // ...
  }

  connectedCallback() {
    this.addEventListener('click', this::_onClick);
  }

  disconnectedCallback() {
    // Assuming this::_onClick === this::_onClick
    this.removeEventListener('click', this::_onClick);
  }
}

which would be pretty nice and intuitive.

[gilbert]

Nicely done! The loss of terse syntax for unary and curried functions is arguably significant, but in return you get pretty much everything else one could ask for 😄

Some more fun features of this syntax:

let lastItem =
  getArray() |> $[$.length-1]

let neighbors =
  array.indexOf('abc') |> array.slice($ - 1, $ + 2)

let silly =
  f |> $(x, y)

[dead-claudia]

This looks a lot like a variant of the partial application proposal, just using a $ (a valid identifier) instead of a ? (which is what that proposal uses).

Maybe we could take some of the features proposed here and integrate them into that proposal, but from the surface, this looks flat out identical to that proposal mod the change in syntactic marker.

[mAAdhaTTah]

I guess I'm surprised $ would even be possible. Something like the $.map example would be especially confusing because jQuery has a $.map. Could the grammar specify whether that's supposed to call immediately or not? If someone did this:

const $ = require('ramda') // using ramda cuz I'm familiar

[1, 2] |> $.map(x => x + 1)

it can't tell when it should use the return value of Ramda's curried map or the Array.prototype.map. Is using a valid variable name explicitly intended?

@isiahmeadows My understanding is this proposal explicitly evolved out of prior attempts to combine the two proposals.

Given that, though is the intention to actually combine the proposals or use the placeholder semantics defined as part of the pipeline proposal as a springboard for adding partial application to the language generally?

[js-choi]

I'm not actually proposing this, but, for what it's worth, the Clojure programming language has a similar pipelining feature called the as-> macro. It has a somewhat different approach, though: it requires explicitly specifying the placeholder argument at the beginning of the pipeline—which may be any valid identifier.

That is, the example in the original post would become something like:

anArray |> ($) { // The $ may be any arbitrary identifier
  pickEveryN($, 2),
  $.filter(...),
  shuffle($),
  $.map(...),
  Promise.all($),
  await $,
  $.forEach(console.log)
}

…but the $ could instead be _ or x or ス or any other valid identifier. That's the advantage: It doesn't implicitly clobber a specially privileged identifier; it requires an explicit identifier for its placeholder argument binding. The downside is that it needs to have its own block—which would contain a list of expressions through which to pipe placeholder data, and within which the placeholder identifier would resolve—rather than the current |> binary operator. That's probably not tenable, I'm guessing.

Having said all that, like @isiahmeadows I too am wondering if ?, as a nonidentifier, would not be a better choice here. The relationship between this proposal and the partial-application proposal may need to be once again considered.

[JAForbes]

I see the benefit of this for multi arg functions, but I don't understand why we need a placeholder for single arg functions? A lot of the FP community in JS has moved or is moving to unary only functions, and so for us it'd be quite natural to do x |> f |> g |> h, littering that expression with $ (or some other symbol) for every application would be pretty noisy, so I'd like to know the benefit.

[JAForbes]

I'm also really surprised await is even a part of this proposal.

anArray
  |> pickEveryN($, 2)
  |> $.filter(...)
  |> shuffle($)
  |> $.map(...)
  |> Promise.all($)
  |> await $
  |> $.forEach(console.log);

Should just be with lodash/fp, ramda, sanctuary ( etc )

anArray
  |> pickEveryN(2)
  |> filter(f)
  |> shuffle
  |> map(f)
  |> Promise.all
  |> then( forEach(console.log) )

|> is a very simple operator, it just applies a function with the result of the previous value, it doesn't need any other semantics.

If we want to use legacy OO style libraries with this syntax, we should have a separate proposal (like the placeholder proposal), and it should not be pipeline specific, it should be language wide. The worst case scenario with arrow functions is completely fine. And the fact it's not as convenient as unary functions is a good thing, it's a subtle encouragement to think compositionally and design API's that enable that.

E.g. let's say we're using pipeline with no placeholder, with no await, with standard lodash and bluebird.

anArray
  |> x => pickEveryN(x, 2)
  |> x => filter(x, f)
  |> shuffle
  |> x => map(x, f)
  |> Promise.all
  |> then( xs => xs.forEach(console.log) )

It's not bad for an API that is not designed for unary application.

I think we may be over-complicating things to support functionality we don't need to support. The work-around of using arrow functions is still simple to follow, and any special placeholder syntax should be language wide anyway and probably beyond the scope of this proposal.

[mAAdhaTTah]

Unfortunately, latest guidance from TC39 is they want await support. This is a syntax that will need to be solved before it can advance.

[TehShrike]

Is nobody at TC39 willing to oppose await in this proposal? It seems like a bad idea to take this super-handy operator common to other languages and bolt on this awkward feature.

[TehShrike]

To be fair, I haven't heard the discussion, so maybe I'm missing something?

[TehShrike]

The notes from the discussion seem more negative towards the idea of bolting on await/yield than anything http://tc39.github.io/tc39-notes/2017-11_nov-29.html#interaction-of-await-and-pipeline

[JAForbes]

Yeah I agree, and the arguments for await are either not justifications or they are reaching for exotic scenarios.

KCL: There are valid usecases for await expressions in a pipeline, an async function that returns a function could be used here.

This is supported by the above |> then( f => f(x) ). Also if exotic examples are required to justify this syntax, then it's not really justifiable.

DE: Yes, I think it'd be very strange if we didn't allow this.

I think we need more guidance than this.

Its a very simple operator, we don't need to complicate it. I think @zenparsing's suggestion is an elegant solution for the constraints tc39 have provided, but I don't think the constraints have any merit and we should seek further guidance.

If we ship it with await banned, and then people want to add await later if/when there's a legitimate justification, we can. It won't break any code. But I personally doubt there will be a justification, await solves a different set of problems.

I'd be in favour of some static functions on the Promise object (e.g. Promise.then ) and ban await for now with an option to add it later if there's ever a need.

[bakkot]

I don't think this is the right thread to revisit await. As a committee member, though, I agree with the assessment that it's something this proposal needs to handle if it's to get through; if you want to discuss that further, I think a new issue would be appropriate.

[mAAdhaTTah]

Personally, I think part of what makes the pipeline operator so elegant is its simplicity. I'd be extremely happy simply writing code like this:

anArray
  |> x => pickEveryN(x, 2)
  |> x => filter(x, f)
  |> shuffle
  |> x => map(x, f)
  |> Promise.all
  |> then(xs => xs.forEach(console.log))

This builds on syntax I'm already familiar and comfortable with (arrow functions), themselves already beautifully simple and elegant, and it feels like a natural evolution for them. Introducing both the |> and a pipeline-only placeholder syntax feels like... too much. I don't think the extra syntax is needed, and I think we'd be better off treating that as a different issue.

I still think I prefer inlining await into the pipeline, despite ASI hazards, but in the interest in exhausting all options, what about maintaining the elegance of the current approach, but use the placeholder syntax to "solve" await:

anArray
  |> x => pickEveryN(x, 2)
  |> x => filter(x, f)
  |> shuffle
  |> x => map(x, f)
  |> Promise.all
  |> await ?
  |> xs => xs.forEach(console.log)

such that the placeholder is only used in await in a pipeline? For the most common case (sync pipelining), we still get a great syntax with arrow functions, and still looks pretty clean in the async case, and perhaps could be useful as a "foot in the door" for placeholders without requiring fully implementing that proposal, which would give developers an opportunity to start developing an intuition for it.

Thoughts?

[zenparsing]

@JAForbes Javascript is a big, huge tent, filled with many people that don't consider OOP "legacy". We need to consider everyone under that tent when designing syntax.

@mAAdhaTTah I don't think we can assume yet that "arrow functions without parens" syntax is going to work out: see https://github.com/tc39/proposal-pipeline-operator/pull/70#issuecomment-359418777. If we have to rewrite your example with the parentheses, it doesn't look as attractive:

anArray
  |> (x => pickEveryN(x, 2))
  |> (x => filter(x, f))
  |> shuffle
  |> (x => map(x, f))
  |> Promise.all
  |> await
  |> (xs => xs.forEach(console.log))

[JAForbes]

@zenparsing

Javascript is a big, huge tent, filled with many people that don't consider OOP "legacy". We need to consider everyone under that tent when designing syntax.

Let me clarify what I mean by legacy. I don't mean legacy today, I mean legacy after this feature lands and is adopted by the community.

This feature will be so transformative for the community, it's hard to over estimate. Currently if you want a library to support some functionality you need to open a pull request and add a method to their prototype. This feature allows us to do all of that in user land by just piping their library into our own functions.

Inevitably that will happen.

That is going to affect the status quo, and we should design for that reality not for what is currently normal.

If we have to rewrite your example with the parentheses, it doesn't look as attractive:

I think it still looks fine, and it keeps this language feature simple. Let's keep the semantics simple and ship it.

[zenparsing]

@JAForbes Can you provide an example, using the semantics of this proposal, that shows how it is unergonomic for some use cases?

[bakkot]

I don't mean legacy today, I mean legacy after this feature lands and is adopted by the community.

I really don't think this feature will drive OO programing out of JavaScript.

Currently if you want a library to support some functionality you need to open a pull request and add a method to their prototype.

See the interfaces proposal for a different approach to this problem.

[JAForbes]

@bakkot I didn't say it would drive OO programming out of JS. I'm saying the release of this operator will change ecosystem norms for data transforms. You can still have classes, and OO and all that stuff, but for data transformation |> will have a massive affect on the community. Designing for an ecosystem that doesn't have |> yet is a mistake. Libraries will change, assumptions will change, patterns will be built entirely around this feature because it's that much of a big deal.

E.g. waiting for Array utilities to land becomes less of a concern, we can do that in userland. Or waiting for Promise, Observable, Iterator functions etc. We can add features immediately and get the same convenience we are used to with chaining. It relieves pressure on library authors, on language designers, it gives more power to the community. We're going to see the pipeline being used in all kind of innovative contexts e.g. decorating hyperscript and jsx. It's hard to predict beyond that, but this feature is a huge deal and it will have drastic affects on the JS ecosystem.

So let's design for that. Just supporting the core functionality will bring a lot of value with very little risk. I'm not at all saying OO will become legacy. I get why you are reading what I'm saying that way, and I apologise sincerely for being unclear but that is not what I am trying to communicate at all. I just want to make sure we don't bake in things that make the feature confusing for an imaginary use case when a simpler proposal provides so much value and is so much easier to teach and adopt.

Designing languages is like designing infrastructure for a city. You need to plan for the future. Future economies, future culture, future populations, future needs. I'm saying, let's not bind ourselves to our assumptions about usage based on the present day, let's keep our options open and add affordances for await and methods if the community needs it. In the mean time, arrow functions work just fine and there's a whole suite of libraries that are made to work perfectly with this feature. Let's ship the simplest possible operator, and give ourselves room to extend if we need it.

[JAForbes]

@zenparsing

Can you provide an example, using the semantics of this proposal, that shows how it is unergonomic for some use cases?

I think your proposal is a brilliant design for the constraints that have been presented. It really is elegant. But I want to ensure that there is a legitimate need for a compromise.

The common case for pipeline execution is composing sync functions. This proposal solves for a lot of contexts but sacrifices the absolute simplicity of x |> f |> g |> h, and instead it becomes x |> f($) |> g($) |> h($). It's not at all unreadable, or unergonomic. But its crossing a threshold of simplicity that I would want to avoid unless it's absolutely necessary. It makes it that much scarier for beginners, that much more intimidating.

If it turns out in #86 there are some solid justifications, I'd support your proposal because I think it's probably the best compromise that could be concocted. But I don't want to compromise without justification. And I'd really like us to ship the simplest possible operator.

[gilbert]

But its crossing a threshold of simplicity that I would want to avoid unless it's absolutely necessary.

I think you need to define what you mean here. "Simple" is often treated as an always-good attribute instead of the tradeoff that it truly is. Aren't function parameters simple? Then why complicated them with default values? Aren't variable assignments simple? Then why complicate them with destructuring? Aren't script tags simple? Then why complicate things with modules?

Reconsidering the meaning of the word, both the original and this Hack-style proposal are equally simple. They both have a simple rule: "call the right-hand side" and "create a binding for the right-hand side", respectively. It just so happens that Hack-style is more ergonomic for a much broader variety of use cases, but at the cost of being more verbose for the simplest use case (unary function calls) Edit: and also curried functions, to be fair.

[gilbert]

Going back to pointing out issues with this proposal, what happens when the temporary identifier is not used on the right hand side? Example:

var result = x |> f($) |> g

Should this result in a syntax error, or would result now point to the function g? Or even crazier, would the parser detect this and invoke g like in the original proposal?

I think this is important to address since I imagine some will forget to pass $ into a righthand expression.

[bakkot]

I agree with the concerns about the choice of binding.

That aside, I want to point out another nice synthesis (with a proposed feature): this works nicely with do expressions. For example:

anArray
 |> pickEveryN($, 2)
 |> do { let [fst, snd, ...rest] = $; [snd, fst, ...rest]; } 
 |> $.map(whatever)

Maybe that do would be better extracted to a function, but I think that's true of most uses of do expressions, so whatever.

[JAForbes]

@gilbert

I think you need to define what you mean here.

Yeah good point. I think this proposal is simple in the abstract. It's design is simple. But it's design also inherits the complexity of the surrounding language, unlike a traditional |> which only can compose functions.

Additionally in terms of actual engine implementation and in terms of the things a user can do with this new syntax, it's more complex. There will be reams of edge cases and blog posts demystifying it's use in corner cases, because it's wide open in what it permits. There'll be deopts for years to come because it will be so hard to predict behavior. Where as in lining a composition pipeline is far more straight forward.

You are right simplicity is a trade off. I think this design is elegant given the constraints. But that's why I'm questioning the justification for those constraints in #86.

I also do not want to have to include a placeholder for every composition, particular when my functions are already unary anyway and I use composition exhaustively. I'll have ? all over my code base to support a use case I don't have, and that I'm not convinced is a good idea anyway. It feels like a repeat of A+ where we forever inherit awkward code because of a supposed need the community didn't ask for.

Yes shipping |> without await may inhibit future designs, but it doesn't rule out the possibility of supporting await. Especially if await expressions are banned from pipeline in the interim.

[dead-claudia]

Edit: Made a few small clarifications.

Here's my thoughts on await:

1. Why special case it to partial application, if it only works within async functions?
2. It could easily be "shimmed"* for non-async contexts through const then = f => async (...args) => f(...args), although this wouldn't translate well to things like generators/etc.

Now, to resolve 1, you could alter await behavior to work this way:

• x |> (await f) does what you'd expect: it awaits f and invokes x with the awaited result.
  • Parentheses are required if you want to directly call the previously awaited result.
• x |> await f is equivalent to await f(x).
  • This keeps partial application separate from the pipeline operator.
• x |> await f is only valid within async functions for obvious reasons.

I know that complicates a potential use case, but it does so in favor for another more frequent one.

Similarly, I'll propose a few more useful additions:

• x |> yield f is equivalent to yield f(x), available in generators only.
• x |> yield* f is equivalent to yield* f(x), available in generators only.
• x |> do ..., etc. perform the right hand side (whether it be a function, await, yield, or yield*), but return the argument passed in, not the result.
  • This lets you do things like call a void function or yield a result mid-flight, while still being able to continue the pipeline with the same value.
    • This would come in handy with array manipulation and DOM manipulation.
  • There's substantial precedent in libraries:
    • Consider the common tap function/method available in numerous utility libraries (like Lodash, RxJS, Underscore, Bluebird, among others).
    • A lot of each utility functions return the instance iterated.
    • Consider the vast number of fluent chaining APIs (notably jQuery here).
  • If you've got a better keyword than do, I'm flexible. (There is void, a little-known keyword that could see a little more use, but that's two more characters I have to type...)
  • Note: this can only be implemented as a function if await/yield/etc. aren't available within pipelines.

---

* I mean this exceptionally loosely, in the sense that es5-sham "shims" Object.getOwnPropertyDescriptor.

[zenparsing]

@isiahmeadows

The "Hack-style" proposal here isn't trying to introduce any kind of partial application to the pipeline operator. Under this "Hack-style" proposal, the pipe operator simply creates a constant variable binding for the left operand when evaluating the right operand. Also, see #83 for a longer discussion on the await syntax as it fits into the current (implicit-call) semantics.

@JAForbes

I'll have ? all over my code base to support a use case I don't have

But other people will have those use cases. Should we make the syntax unergonomic/confusing for them?

Also, are we sure that we even need syntax for the implicit-call semantics? If you are really using composition for everything, can we use a function for that?

let result = purePipe(anArray,
  pickEveryN(2),
  filter(f),
  shuffle,
  map(f),
  Promise::all, // Future method-extraction syntax
  then(xs => xs.forEach(console.log)),
);

You could even combine that with |>:

anArray 
  |> purePipe($)
  |> $( pickEveryN(2),
        filter(f),
        shuffle,
        map(f) )
  |> Promise.all($)
  |> await $
  |> $.forEach(console.log);

[zenparsing]

@gilbert

I think this is important to address since I imagine some will forget to pass $ into a righthand expression.

For this reason I would say that a syntax error is probably the best option.

[mAAdhaTTah]

Is a placeholder required for unary functions? Is there a reason we couldn't allow mixed-use? Modified example from OP w/ ? instead:

anArray
  |> pickEveryN(?, 2)
  |> ?.filter(...)
  |> shuffle // Doesn't require the ?
  |> ?.map(...)
  |> Promise.all // neither does this
  |> await ? // maybe even drop it here?
  |> ?.forEach(console.log);

Are we mostly concerned here about developer confusion?

[zenparsing]

@bakkot

I agree with the concerns about the choice of binding.

Yes, I don't think "$" is going to be the best option, but I wanted to get the discussion going.

Leaving jQuery aside, I think that the binding name should really stand out to readers (especially so since the declaration is implicit). The "$" symbol does stand out from other identifiers but I'm worried that it will get lost in the mix if the RHS contains template literals. As a trivial example:

"Earth" |> `Hello ${$}`;

I'm interested in seeing other ideas for the binding name!

[zenparsing]

@mAAdhaTTah

Are we mostly concerned here about developer confusion?

Yes, that and the potential for the compiler to wrongly guess the user's intention.

Here's your version with a slight twist:

anArray
  |> pickEveryN(?, 2)
  |> ?.filter(...)
  |> shuffle(9)
  |> ?.map(...)
  |> Promise.all
  |> await ?
  |> ?.forEach(console.log);

Is the intention to call shuffle(9)(?)? Or did the user mean to say shuffle(?, 9). Or even shuffle(?)?

[js-choi]

@zenparsing With regard to implicit-call pipelining and the purePipe idea, the Clojure programming language has precedent for separating explicit- from implicit-call semantics into multiple forms.

I referred above to its as-> macro, which requires explicit declaration of a variable binding. Clojure also has -> and ->> macros, which implicitly add an argument to the start or end of a function’s parameters, respectively. And just like in the last example in https://github.com/tc39/proposal-pipeline-operator/issues/84#issuecomment-359828582, ->, ->>, and as-> can be arbitrarily nested.

However, Clojure’s -> and ->> forms cannot be runtime-evaluated functions, and the purePipe idea in https://github.com/tc39/proposal-pipeline-operator/issues/84#issuecomment-359828582 is a function, rather than syntax. From what I can tell, purePipe would have to create and return a function at runtime. And it requires each of its arguments to evaluate into functions, too: pickEveryN would need to support currying, and pickEveryN(2) must return a function. That may be easier with the proposal for syntactic partial application, but it still has the drawback of unnecessarily creating function objects, which is a big raison d’etre for this proposal.

I’m still not proposing that JavaScript follow Clojure’s model, but Clojure’s separation of implicit- and explicit-call pipelining does offer flexibility. If we say Clojure’s as-> and -> respectively become |> in and |> prefix operators in JS, then the two examples in https://github.com/tc39/proposal-pipeline-operator/issues/84#issuecomment-359828582 would become:

// does not require pickEveryN(2), filter(f), etc. to create and return functions
let result = anArray |> {
  pickEveryN(2),
  filter(f),
  shuffle,
  map(f),
  Promise::all,
  then(xs => xs.forEach(console.log)),
};

anArray |> in ($) {
  $ |> {
    pickEveryN(2),
    filter(f),
    shuffle,
    map(f)
  },
  Promise.all($),
  await $,
  $.forEach(console.log)
}

…But, like I said in https://github.com/tc39/proposal-pipeline-operator/issues/84#issuecomment-359608199, these probably would require turning |> into a prefix operator acting on lists of expressions, rather than a binary operator interposed between each pair of expressions. That would be the advantage and disadvantages of Clojure’s Lisp syntax, and it probably would not be tenable in JavaScript.

There might be a way to make the syntax above into binary operations elegantly, but I don’t see a way. Especially given the fact that part of its point is that the placeholder variable is explicitly declared to be $ or any othe ridentifier, rather than a particular syntactically privileged identifier. I just wanted to bring up the fact that there is precedent in separating implicit- from explicit-call pipelining semantics in at least one other language.

[js-choi]

…Continuing https://github.com/tc39/proposal-pipeline-operator/issues/84#issuecomment-359839978, come to think of it, if ? were used instead of an explicitly bound identifier in the explicit-call syntax, then the examples in https://github.com/tc39/proposal-pipeline-operator/issues/84#issuecomment-359828582 could instead become:

// Still does not require pickEveryN(2), filter(f), etc. to return functions
let result = anArray
  |>> pickEveryN(2)
  |>> filter(f)
  |>> shuffle
  |>> map(f)
  |>> Promise::all
  |> ?.then(xs => xs.forEach(console.log));

anArray 
  |>> pickEveryN(2)
  |>> filter(f)
  |>> shuffle
  |>> map(f)
  |> Promise.all(?)
  |> await ?
  |> ?.forEach(console.log);

…where |> is an explicit-call pipelining binary operator using ? as an explicit placeholder variable, and |>> is an implicit-call pipelining binary operator that implicitly calls its right side with the left side as its first argument.

I think I like this, personally. By enabling explicit-call semantics it takes care of the await problem without making a special exception for it. By separating implicit-call semantics into another operator it takes care of unnecessary verbosity when you just need to pipe data through first parameters. And by using ? it prevents the explicit-call syntax from being too verbose without privileging a special-but-already-valid identifier like $. There’s hardly any magic here.

[zenparsing]

@js-choi That's funny, I was just about to type in (more or less) the same suggestion for |>>.

[kurtmilam]

@zenparsing

Is the intention to call shuffle(9)(?)? Or did the user mean to say shuffle(?, 9). Or even shuffle(?)?

I don't see any reason for confusion here. I would expect |> shuffle(9) to call |> shuffle(9)(?).

I'd be very disappointed if the efforts to force support for await and offer a shorter replacement syntax for arrow functions came at the cost of removing implicit calls where explicit calls would otherwise be unnecessary.

[zenparsing]

@kurtmilam

I would expect |> shuffle(9) to call |> shuffle(9)(?).

You may, but Javascript is a big tent filled with lots of developers who may not find things like shuffle(9)(x) particularly intuitive or obvious.

It might be helpful if we could see a longer example demonstrating how the Hack-style proposal is burdensome for the "all single arg functions" use case.

[ljharb]

fwiw, i would not expect that. Haskell-style auto-currying is something i find wildly unintuitive.

[mAAdhaTTah]

I wouldn't see this as being an "auto" currying in the |> shuffle(9) case. My interpretation of that is shuffle is a function that returns another function, e.g.:

const shuffle = seed => arr => { /* logic */ }

Thus, doing |> shuffle(9) isn't doing anything special. It just calls the function, and the return value gets slotted into the pipeline. If shuffle wasn't curried like this, e.g.:

const shuffle = (seed, arr) => { /* logic */ }

then the function itself would (presumably) error because it's only being executed with one of the two required arguments, or the pipeline would break because the unexpected return value is not a function, which would cause a TypeError (I think?). In both of these cases, the cause of the problem is clear to the developer and local to the problem area, so I don't think it would be too confusing to developers. Ultimately, without the ?, the function call behaves as it would anywhere else in the code.

If you wanted the function to be curried, you'd have to do so explicitly, using the placeholder. So if shuffle was the second version, you'd have to write shuffle(9, ?) explicitly.

FWIW, I think this is what @kurtmilam intended when he said it would look like shuffle(9)(?); the return value of shuffle(9) is what gets pipelined.

[kurtmilam]

@zenparsing

Javascript is a big tent filled with lots of developers who may not find things like shuffle(9)(x) particularly intuitive or obvious.

My suggestion would be to enable developers who are comfortable with that syntax (i.e. comfortable with a functional programming style) to use it while allowing developers who aren't comfortable with that syntax to eschew it altogether.

It might be helpful if we could see a longer example demonstrating how the Hack-style proposal is burdensome for the "all single arg functions" use case.

Here's a simplified example of an actual pipeline that I have written (converted from my own pipe implementation to the pipeline operator syntax):

const result = array
  >| flatten
  >| filter(isStr)
  >| prepend('none')
  >| last
  >| replace(/[\W_]+/g)(' ')
  >| trim
  >| split(' ')
  >| map(uCFirst)
  >| join(' ')

I use this concept heavily, and it would be burdensome to have to hew to the Hack-style syntax in all the places I use it.

@ljharb My expectation matches the babel implementation, the pipe implementations offered by Ramda and SanctuaryJS and the ones I have written.

What a developer should expect is something that must be defined, but I am asking for this concept, which is brought over from functional programming languages and libraries, to at least remain ergonomic for JavaScript programmers who are comfortable working in a functional style.

JavaScript programmers who are not comfortable with that style would still have the option to explicitly control calling by using placeholders or bound variable constants everywhere.

[gilbert]

But it's design also inherits the complexity of the surrounding language, unlike a traditional |> which only can compose functions.

Additionally in terms of actual engine implementation and in terms of the things a user can do with this new syntax, it's more complex. There will be reams of edge cases and blog posts demystifying it's use in corner cases, because it's wide open in what it permits. There'll be deopts for years to come because it will be so hard to predict behavior. Where as in lining a composition pipeline is far more straight forward.

I'm sorry, but this simplicity argument is really starting to feel like a stretch. You're basically saying if a feature is more general, it "inherits the complexity of the language". That if a feature has more use cases, "it's more complex".

I'm not buying it. How is Hack-style hard to predict behavior? What corner cases are there to demystify? It's literally just a temporary variable. If you know how to use a variable, you know how to use this version of a pipeline. There's nothing "simpler" than that.

[gilbert]

Here's a comparison of the two styles. I did not mention await to demonstrate how that's not the only reason to consider a Hack-style pipeline operator.

✓ = ergonomic, ✗ = not ergonomic, ~ = neutral

Hack-style

✓ Multi-arg Functions x |> f(10,$) ✓ Property access x |> f |> $.key ✓ Property lookup x |> f |> cache[$.id] || find($.id) ✓ Method invocation x |> f |> $.concat(y) ✓ Multiple reference x |> f($, $) ✓ Deeper expressions x |> f(g($)) ✓ Template literals x |> `Hello ${$}` ✓ Object spread (with no arrow fn parens) x |> { ...$, y: 20 } ✓ All Explicit ~ Unary functions x |> f($) ✗ Curried functions x |> map(f)($)

F#-style

✓ Unary functions x |> f ✓ Curried functions x |> map(f) ✗ Multi-arg Functions x |> ($ => f(10,$)) ✗ etc. (all other cases require an arrow function or a library / self-written curried function)

I agree with others that having a |>> alongside |> to make implicit calls would be great. Operator precedence might be a concern, though.

[zenparsing]

I agree with others that having a |>> alongside |> to make implicit calls would be great. Operator precedence might be a concern, though.

I'm thinking it would be another production of PipelineExpression (and have the same precedence). Do you see any issues with that?

PipelineExpression :
  LogicalORExpression
  PipelineExpression |> LogicalORExpression
  PipelineExpression |>> LogicalORExpression

What would we do about await expressions on the RHS of |>>?

x |>> await y;

It should probably just do the naive thing and await y.

[gilbert]

I'm thinking it would be another production of PipelineExpression (and have the same precedence). Do you see any issues with that?

I'm not too familiar with JS grammar, but that sounds like a good idea :)

What would we do about await expressions on the RHS of |>>?

Intuitively I would expect x |>> await y to be parsed as (await y)(x).

[kurtmilam]

@gilbert I don't understand some of the items in your comparison. For instance, under F#-style:

✗ Multi-arg Functions x |> ($ => f(10,$))

Does the '✗' mean the example wouldn't work? According to my interpretation of the proposal (as currently written), it would work as I'd expect it to. The same goes for all of the Hack-Style examples, I believe (i.e. those examples would work with an F#-style pipeline operator if we're allowed to use arrow functions enclosed in parentheses as the proposal currently states). Could you clarify?

I agree with others that having a |>> alongside |> to make implicit calls would be great. Operator precedence might be a concern, though.

This looks like a good compromise solution that I could get behind.

[gilbert]

Sorry, I will edit to provide a legend. '✗' means not ergonomic.

[gilbert]

Another issue to consider. How would Hack-style |> interact with inner function scopes?

var result = 10
  |> function () { return $ |> $+1 }
  |> $()

result //=> ???

Should $ be restricted to the current scope only, to prevent closures like these?

[mAAdhaTTah]

I'm sorry, but this simplicity argument is really starting to feel like a stretch.

From my perspective, this proposal loses simplicity because of how much new syntax a developer has to learn in order to understand it. Initially, we were looking at a single piece of syntax (sigil? not sure if that's the correct word?), |>, that was pretty clear and fairly well-defined (parenthesized arrow functions notwithstanding). Now we're looking at introducing a second sigil, ? or $, for placeholders, and now potentially a third, |>>, if I'm understanding the proposal outlined here correctly.

So it's a lot more syntax, with distinctions for the usage of |>> compared to |> that aren't really clear to me, for what is, at core, a pretty simple idea. I'm not convinced using arrow functions, even with parens, is unergonomic, or at least, unergonomic enough to require working around, and the major benefit is that it builds on syntax developers already know or in the process of learning, which we may be under-appreciating here.

I'm more concerned that adding all this syntactical overhead to using the pipeline makes it less likely that developers will learn it; that we're prioritizing the pipeline's ergonomy (is that a word?) over its learnability, so I'm coming to the opposite conclusion @zenparsing did in #82.

[aikeru]

I've been eagerly awaiting the result of either this proposal, or the :: operator proposal and hoping that one of them will make it far enough so that tools like Flow/TypeScript/WebStorm/etc. can adopt it.

I'd just like to speak up and say that Gilbert's Hack-style examples are very relevant to me and the way I'd like to use this operator. When I saw this issue pop up with the idea of using a lexically-bound variable, I was surprised and delighted at the idea that I wouldn't need a bunch of intermediate arrow functions (or more likely, curry-able wrappers).

F# is a wonderful language, and F# style magical currying strikes me as brilliant, but foreign and strange in JavaScript (even more so due to F# being statically typed vs. JS being dynamic). Hack style currying appears much more natural and flexible to me in JS land. Once the operator is in, it's likely it can't be changed afterward, so I'm hoping for the more flexible version to get standardized.

[zenparsing]

@gilbert

This is a silly example, but I can imagine someone wanting to use the binding in a closure:

let timer = planets
  |> pickRandom($)
  |> "Hello " + $
  |> setTimeout(() => console.log($), 1000);

Hopefully with the right choice of identifier/glyph it will be clear to users where all of the pipe values are.

[js-choi]

@gilbert: With regard to the problem of inner function scopes, looking at the Clojure programming language again may be useful, since Clojure had to deal with similar problems in two ways:

As mentioned above, Clojure’s as-> requires that the placeholder variable be explicitly named. This simply creates a lexical scope for each step of the pipe (acting essentially as an identity monad). This lexical scope can be locally overridden within any inner lexical scope, such as in an inner function, just like for nested lexical scopes in general. Therefore, if an as-> using a temporary variable $ contains another as-> that also binds to $, then $ will refer to the inner as->’s binding within the inner as->’s lexical scope only.

However, as also mentioned, this mostly works because as-> acts as something like a prefix operator, rather than the binary operator that we probably want |> to be. If |> is to be a binary operator, then having to declare a placeholder would be too verbose.

Alternatively, Clojure syntax does have another similar feature: its #(…) form is an anonymous-function shorthand that has implicit parameter variables named %. (N.b. % is not technically a valid identifier character in Clojure, unlike $ in JavaScript…though actually the Clojure compiler currently accepts % as an identifier outside of #(…) without complaining, and that hasn’t been a problem for a decade.)

The important part here is that, like this proposal’s |> and $, Clojure’s #(…) creates implicit parameter-variable bindings to %. This has a problem with nesting: when #(…) contains another #(…), to which function’s argument would % refer? Clojure solved this by syntactically forbidding nested #(…).

So for one manifestation of this nested-scope problem (nested as->s with explicit bindings to arbitrary identifiers), Clojure’s solution says that inner scopes simply lexically override outer scopes, and 10 |> function () { return $ |> $+1 } |> $() would be function () { return 10 |> $+1 } |> $(), i.e. 10 |> $+1, i.e., 11. For another manifestation (nested #(…)s with implicit bindings to a special symbol %), Clojure punts and says that the possible semantic ambiguity to humans isn’t worth allowing nesting the scopes, and 10 |> function () { return $ |> $+1 } |> $() would be a syntax error.

I think that the first solution is obviously better than the second. As @zenparsing points out, nested functions in pipelines is useful; indeed, Clojure programmers use them all the time. But, as long as we are talking about an implicitly bound variable, I’m still wondering whether ? or another non-identifier would be better than an already-valid-identifier like $. Clojure does not have an answer for that, other than that it chose a technically nonvalid but compiler-valid identifier % for an implicit-binding form, forbade nested implicit binding, and has required explicit binding for other syntaxes. Edit: This paragraph’s conclusion is incomplete. The real problem is not RHS nested inner functions; it is RHS nested inner explicit-call pipelines. There is therefore a third option, which allows both callbacks using inner functions but prevents lexical clobbering of the implicit placeholder variable: simply prohibit nested explicit-call pipelines but allow function expressions. See https://github.com/tc39/proposal-pipeline-operator/issues/84#issuecomment-360681313.

TL;DR: If you used Clojure’s as-> (and identity monads in general) as precedent, 10 |> function () { return $ |> $+1 } |> $() would be function () { return 10 |> $+1 } |> $(), i.e. 10 |> $+1, i.e., 11.

---

@mAAdhaTTah I wouldn’t consider this proposal’s $ to be a separate, second piece of syntax from |>. At least, to a Clojure programmer’s mind, % is inalienably connected to #(…). It’s not an indicator character so much as it’s just a regular variable that happens to have an implicit binding within a certain lexical context.

Now, a |>> would indeed be a second syntax indicator, but that’s because it’s for a different purpose: implicit-call pipelining. Clojure programmers use #(…) and % syntax with the ->, ->>, and as-> macros all the time for their functional programming, they are all useful for different data flows, and their differences are easy to reason about. I’m not saying that Clojure is the only functional programming language, but I am saying that it has encountered these problems and created interesting solutions. It doesn’t only have F# (its -> macro); it also has Hack-style (as->). In fact, I personally use as-> much more than ->, despite its slightly increased verbosity…