Closed zenparsing closed 3 years ago
Btw, I am surprised first-argument injection (or last-argument injection) is rejected. Many languages use that mechanism for their pipeline operator. I understand it might feel a little ad-hoc, but considering currying is not a native feature of Javascript, it might be more compatible with the rest of the eco-system. I probably need to dig deeper into that discussion.
BTW, if anyone is using the Hack or Smart style pipelines in their codebase, would love to see some examples how you're using it. Part of my issue is I'm generally seeing adoption within FP communities of F#-style pipelines and have very little visibility into how Hack-style pipelines are being used. Feel free to email me if you're more comfortable w/ that as well (email is in my GitHub bio).
@highmountaintea See #143 & #20 for some exploration of those approaches.
@highmountaintea See #143 & #20 for some exploration of those approaches.
Hmmm, I half suspected as much regarding the incompatibility with curried functions. I'll reserve my comments either in the appropriate threads or until they are applicable to the Hack proposal.
I keep bouncing back and forth between my preferences reading all these great arguments! At first I was Team F# because of the simplicity of it. Then I was Team Hack after considering that most JS code in the wild is not FP-style, and would be more cumbersome to try to write out arrow functions for every pipeline. Now I'm back on F# after considering that Hack would be worse for FP-style than F# would be for non-FP.
Fwiw, the primary use case I'm excited about isn't elegantly handled by either option - I like the idea of writing "extension methods" that apply to certain types of objects and can be used like methods, but aren't actually part of the prototype and can be imported. For F#, those functions would pretty much have to be curried - at least the first argument - which is a departure from the kind of JS I write today. For Hack, it doesn't even really feel like an extension method anymore; just another way of writing a function call... but still a better one for many use cases than we have today.
@dallonf I've been arguing that "extension methods" (what I, in retrospect, erroneously called "importable methods" elsewhere) will look closer to what your run-of-the-mill fluent API looks like with F#:
someArray |> map(x => x * 2) |> filter(x => x < 100)
looks awfully close to using .map
& .filter
. While the implementation would have to be curried (depending on what is mean by "curried" exactly), which is a burden on the implementor, but for a consumer, there isn't any conceptual overhead translating from fluent -> F# style pipelines using extension methods, whereas with Hack-style, you'd probably implement them:
someArray |> map(#, x => x * 2) |> filter(#, x => x < 100)
which takes what was previously hidden (the data that was be operated on, handled via this
in fluent APIs & a curried param in F#) and puts it right in front of you. It is my personal belief that should we get a Hack-style pipeline, the community will largely not be adopting it for extension methods for this reason because it is not syntactically advantageous vs current userland implementations.
@dallonf sounds like you are looking for first-argument injection?
@dallonf sounds like you are looking for first-argument injection?
In an ideal world, yes, but since JavaScript really has no formal patterns encouraging the first argument being special in any way, I'd rather have the explicitness of either calling with a single-argument (F#) or explicitly using a placeholder where the the piped argument should go (Hack), even if it's a little less convenient.
It is my personal belief that should we get a Hack-style pipeline, the community will largely not be adopting it for extension methods for this reason because it is not syntactically advantageous vs current userland implementations.
I think this still depends - from another viewpoint, Hack-style pipeline would turn libraries like Lodash (which is possibly a bad example since Lodash does actually have an auto-curried FP version) into essentially collections of extension methods with no change required for the maintainers to support it.
So for the extension use case specifically, it's really a matter of whether you want to put the burden on library maintainers or adopters to actually make the feature streamlined. The fact that several libraries (like Lodash and date-fns) have already done this work does make me lean more towards F#-style though (although I certainly wouldn't say no to Hack if it's between that and no pipeline!).
Also... for currying, there's always array |> flatMap.bind(undefined, (x) => x.y)
... or perhaps even an extension method for functions to auto-curry them! 🙂
@dallonf sounds like you are looking for first-argument injection?
In an ideal world, yes, but since JavaScript really has no formal patterns encouraging the first argument being special in any way, I'd rather have the explicitness of either calling with a single-argument (F#) or explicitly using a placeholder where the the piped argument should go (Hack), even if it's a little less convenient.
Gotcha. I understand your concern is about there is no standard in JS in terms of which argument being special, however, https://github.com/tc39/proposal-pipeline-operator/issues/143#issuecomment-492019845 might still be of interest to you.
Hi @highmountaintea. Could you explain what first-argument injection is? I'm not familiar with that term.
Hi @highmountaintea. Could you explain what first-argument injection is? I'm not familiar with that term.
It first assumes the RHS of a pipeline expression is in the form of a function invocation. You transform the pipeline expression by injecting LHS into the first argument of the RHS. It works well with libraries that assume the first argument to be the topic. Clojure, Elixir and Racketr have this feature. So:
population
-> filterBy(x => x.age > 18)
-> map(x => x.firstName)
is transformed into:
map(filterBy(population, x => x.age > 18), x => x.firstName)
You can read Clojure's thread-first macro for more information: https://clojure.org/guides/threading_macros
Edit: Sorry, I explained it in the context of pipeline operator. In a broader sense, it's the concept of using the subject as the first argument of a function. For example, we can think of the OO form sourceStr.split(separator)
as a sugared form of split(sourceStr, separator)
. Here split()
is actually a function that takes 2 argument, and the first argument is the topic or the subject to be operated on. In a sense, that's how JS operates anyway, as shown by the Function.call()
method:
let split = String.prototype.split;
split.call('Hi there', ' '); // -> ["Hi", "there"]
If you think of how an OO language is typically implemented. Underneath the surface, an OO method is still pretty much just a function. It is combined with a dynamic dispatch to satisfy OO's polymorphism. Because a method is just a function underneath, it needs a way to carry the main object as a parameter of the function. In the case of fullName.split(' ')
, the object fullName
needs to be passed in as a parameter to the underlying function. It is typically implemented as a first argument topic
, so the underlying function invocation is really realSplit(fullName, ' ')
.
* Thus, first argument topic
is natural to Javascript, not an ad-hoc rule invented after the fact.
Only when that injected first argument becomes the receiver (the this
). I wouldn’t expect FP users to be excited about using this
inside their functions, and any other form would not be natural to JavaScript.
Also... for currying, there's always array |> flatMap.bind(undefined, (x) => x.y)... or perhaps even an extension method for functions to auto-curry them! 🙂
This was the idea behind my partial application proposal
Also... for currying, there's always array |> flatMap.bind(undefined, (x) => x.y)... or perhaps even an extension method for functions to auto-curry them! 🙂
This was the idea behind my partial application proposal
I am a fan of your original partial application proposal. But the library you linked here only allows partial application starting at the beginning, instead of allowing holes? e.g. fn(1, ?, 2)
or fn(?, 'xyz')
(just wondering if it's the limitation of the library or it's a change in the proposal itself)
What are you referring to when you say library @highmountaintea ?
What are you referring to when you say library @highmountaintea ?
Hi, I was referring to Function.prototype.papp()
and Function.prototype.pappRight()
mentioned in your link above.
Ah sorry, I got confused because you were using library and proposal interchangeably. Although you could argue that's valid since it'd be part of the JS standard library.
Yes, it only supports partial application from the beginning. As a result, it requires no new syntax.
Although useful on its own, it would be complementary to an F# style pipeline operator.
I want to propose a modified version of Hack proposal. I'll call it Hack Prime because it's based on the well laid out mechanism mentioned at the top of this thread. If this has already been proposed please point it out.
The canonical form of Hack Prime is: PipelineExpression : PipelineExpression ~>(Variable) LogicalORExpression
.
x
, y
, temp
, $
etc.An example of canonical Hack Prime would be:
let ageMap = fetch()
~>($) await $
~>($) filterBy($, x => x.age >= 18 && x.age <= 65)
~>($) sortBy($, x => x.rating, 'desc')
~>($) pickN($, 100)
~>($) mapBy($, x => x.age)
The canonical form is intentionally verbose and precise, but hopefully easy to read. Just like the original Hack proposal, this covers almost all usage cases in a consistent manner.
To make it more ergonomic, we then introduce a short form. Which is of the form: PipelineExpression : PipelineExpression ~>> Function(...)
.
The short form is restrictive in the sense that the RHS has to be of the form of a function invocation. It expands into the canonical form by first-argument injection. Assuming Φ
is an uniquely generated temporary variable name, ~>> Function(...)
expands into ~>(Φ) Function(Φ, ...)
.
So the above code example can also be written as:
let ageMap = fetch()
~>($) await $
~>> filterBy(x => x.age >= 18 && x.age <= 65)
~>> sortBy(x => x.rating, 'desc')
~>> pickN(100)
~>> mapBy(x => x.age)
The difference between the short form and other proposals that have similar syntax is that the short form here relies on the simple yet consistent runtime semantic of the canonical form, and it directly translates into it. This allows a fallback when we need to work with generators and awaits, and retains the versatility of the original Hack proposal. As shown above, we can mix short and canonical forms without problem.
I love this idea of specifying the binding and using it!
I can't say I like the syntax (tilde character in the Portuguese keyboard is also an accent for ã, so we have to press it twice)
The shorthand form is very compelling especially to someone who likes the elixir proposal ;)
It's also easy to minify (at least from Terser's perspective) since we only need to add a new ~> scoped binding and no currying is going on.
The biggest thing I'm taking away from this idea is maybe we need two operators 🙂
I'm wondering if it would be useful to access previous bindings as well, or if it would be more confusing.
await fetch(url)
~>(res) await res.json()
~>(body) [res.ok, body.data]
To me it doesn't feel weird, but I read the above as something like a nested do-expression, and many will disagree for sure.
do {
let res = await fetch(url)
do {
let body = await res.json()
[res.ok, body.data]
}
}
Why do we need new syntax to specify a binding when we have arrow functions?
@mAAdhaTTah
If you mean something like this:
expr
(arg) => doSomething(arg)
Then it's ambiguous whether these are two separate statements, or just one pipeline expression.
But adding something like |>
or ~>
before the arrow function would disambiguate this I guess.
Edit: Just realized that adding |>
before the function is basically what this proposal already is, and will stop bikeshedding now. Either one is cool in my book.
No, I mean, literally the F# proposal:
let ageMap = fetch()
|> await
|>($) => filterBy($, x => x.age >= 18 && x.age <= 65)
|>($) => sortBy($, x => x.rating, 'desc')
|>($) => pickN($, 100)
|>($) => mapBy($, x => x.age)
The whole purpose of the base Hack proposal is using #
(or whatever token is decided) is terser than using arrow functions. If you start adding extra syntax to choose the token (e.g. "Hack Prime"), you lose the primary value of Hack, and you're add a lot of new syntax to do what F# already does "built-in" (because arrow functions are already a thing).
again, this proposal is needless tech-debt, when temp-variable is cleaner and easier to read-and-refactor.
let result;
result = await fetch();
...
result = Array.from(result);
result = result.filter(function (elem) { return elem.age >= 18 && elem.age <= 65 });
result = result.sort(function (aa, bb) {
aa = aa.rating;
bb = bb .rating;
return (
aa < bb
? -1
: aa > bb
? 1
: 0
);
}).reverse();
result = result.slice(0, 100);
result = result.map(function (elem) { return elem.age; });
let result;
result = await fetch(url);
result = {
statusCode: result.statusCode,
ok: result.ok,
responseJson: await result.json()
};
result.responseJson = result.responseJson.data;
your trying to add a feature intended to overcome handicaps in static-languages that have no ergonomic value in javascript.
@kaizhu256 and again, please stop making those kind of comments; reassignment is not a good practice to everyone, and there is a need for many of us, even if you don’t have one.
There is nothing ergonomic about a list of statements that all reassign to the same variable.
yes it is more ergonomic. javascript development is messy with entire codebases frequently rewritten every week (or every day) as a product's ux-wiring and ux-kinks are worked out.
pipeline-operators, and needless abstractions like filterBy, sortBy, ...
when builtin array methods would do just fine, simply get in the way of the endless javascript code-rewrites product-developers have to deal with, as the ux evolves.
There is nothing ergonomic about a list of statements that all reassign to the same variable.
On the other hand, I'm starting to wonder what's bad about dot-chaining. JavaScript is an OO language after all, not a functional one. OO tries to keep functions organized; most pipe examples I see are using global functions.
Why are stream libraries moving from dots to pipe()?
let result;
result = await fetch();
...
result = Array.from(result).
filter(function (elem)
{ return elem.age >= 18 && elem.age <= 65 }).
sort(function (aa, bb) {
aa = aa.rating;
bb = bb.rating;
return aa-bb;
}).
reverse().
slice(0, 100).
map(function (elem) { return elem.age; });
I mean, there are a few cases where a pipe would be useful, in particular when I need the value more than once.
bigfunc(x) |> sendmail(#.subj, #.body)
But it's not like every other line of code would have a pipe in it.
dot-chaining is generally the "right" way to transform arrays and strings in javascript (when builtin methods will suffice).
for the bigfunc(x)
example, again, use a tmp-var:
let result;
result = ...
result = bigfunc(result);
result = await sendmail(result.subj, result.body);
result = ...
above code is easier to refactor than your pipeline example in the endless javascript-churn we all deal with.
@MrVichr I think the main reason for it is tree shaking. From RxJS docs https://v6.rxjs.dev/guide/v6/pipeable-operators
Problems with the patched operators for dot-chaining are:
Any library that imports a patch operator will augment the Observable.prototype for all consumers of that library, creating blind dependencies. If the library removes their usage, they unknowingly break everyone else. With pipeables, you have to import the operators you need into each file you use them in.
Operators patched directly onto the prototype are not "tree-shakeable" by tools like rollup or webpack. Pipeable operators will be as they are just functions pulled in from modules directly.
Unused operators that are being imported in apps cannot be detected reliably by any sort of build tool or lint rule. That means that you might import scan, but stop using it, and it's still being added to your output bundle. With pipeable operators, if you're not using it, a lint rule can pick it up for you.
Functional composition is awesome. Building your own custom operators becomes much easier, and now they work and look just like all other operators in rxjs. You don't need to extend Observable or override lift anymore.
@mAAdhaTTah yes totally agree there!
@kaizhu256 and again, please stop making those kind of comments; reassignment is not a good practice to everyone, and there is a need for many of us, even if you don’t have one.
There is nothing ergonomic about a list of statements that all reassign to the same variable.
I agree. I have tried to use the temp-variable assignment method in practice, and learned quickly that it's not a good replacement for the Hack proposal.
Why do we need new syntax to specify a binding when we have arrow functions?
Hi @mAAdhaTTah,
I added the bind token syntax partially to make the short form easier to define. It also makes the canonical form a little more precise. With the short form, we won't need to use the canonical form as much, so we can afford to be a little more verbose. But you could be right, and people might prefer the convenience of binding it to an implicit token.
There is still some difference between the token-binding syntax here and the F# proposal + arrow functions, because the token-binding syntax here works with generators and awaits, and have no performance penalties.
@highmountaintea I understand the thought process. I'm just pointing out that in doing so, you're reducing/eliminating one of the main advantages Hack has over F# (terseness) while compounding one of its main downside (novel syntax).
I'm wondering if it would be useful to access previous bindings as well, or if it would be more confusing.
await fetch(url) ~>(res) await res.json() ~>(body) [res.ok, body.data]
It's an interesting idea. Seems versatile, but leaking the variable scope into the next pipe might feel like an issue.
It probably is an issue. It's no longer easy to read since you have to go make sure bindings aren't reused below, unless you use the same name in every binding.
No, I mean, literally the F# proposal:
let ageMap = fetch() |> await |>($) => filterBy($, x => x.age >= 18 && x.age <= 65) |>($) => sortBy($, x => x.rating, 'desc') |>($) => pickN($, 100) |>($) => mapBy($, x => x.age)
The whole purpose of the base Hack proposal is using
#
(or whatever token is decided) is terser than using arrow functions. If you start adding extra syntax to choose the token (e.g. "Hack Prime"), you lose the primary value of Hack, and you're add a lot of new syntax to do what F# already does "built-in" (because arrow functions are already a thing).
Why can't we get a champion here that will push the simpliest, minimal, F# proposal forward without this needless spinning our wheels all these years @mAAdhaTTah? We can always build on the minimal foundation as needed.
@aadamsx I've been working on this for 3+ years. Daniel was championing it most of that time and preferred F#. Tab has picked it up now and prefers Hack. The problem is not the champion, it's consensus.
I think this is an instance of the law of triviality at work.
I'm guilty of it too, in this instance and others. But at the end of the day, does it matter, really?
I suspect that in the future we'll have linters and peer pressure stopping us from doing crazy things in pipes. Having worked at an elixir shop I was under strict guidelines on what to do and not to do, because if you overuse the pipe operator things get unreadable pretty fast.
So whatever syntax we choose, people will use it in the same way. It's just a matter of balancing power for crazy devs like me, with how much we need to teach new JS developers to get them up to speed on this syntax.
I've been working on this for 3+ years. Daniel was championing it most of that time and preferred F#. Tab has picked it up now and prefers Hack. The problem is not the champion, it's consensus.
The problem is there will NEVER be a consensus on this! The TC39 guys have also aluded to the process not being based on concensus. So a champion leaning one way or another is the only thing I think that can tip the scales one way or another.
You've been on this from the start, why can't YOU be the champion here @mAAdhaTTah?
I suspect that in the future we'll have linters and peer pressure stopping us from doing crazy things in pipes. Having worked at an elixir shop I was under strict guidelines on what to do and not to do, because if you overuse the pipe operator things get unreadable pretty fast.
Making up for flawed syntax with linters -- not a great idea @fabiosantoscode
TC39 is exclusively based on consensus; I’m not sure where anything else has been alluded to.
Who is to say when we've reached a concensus other than the champion and/or TC39? For example @ljharb, could Tab take champion status over this repo and move forward with Hack in this repo's current state (when we obviously do not have concensus)?
From what I've read on this repo, TC39 takes our input to an extent, but ultimatly the decision is in their hands, and a concensus is not required for them to make a decision on way or another. I'll have to do searches for these types of posts, but don't have that much time today.
Consensus is among tc39 delegates, so yes, if every delegate agreed with a direction despite the concerns of participants on this repo, then it would move forward. However, i think it’s unlikely that the committee would agree to proceed with anything if there remains this much contention here.
In other words, in the absence of a compelling and persuasive argument that can override aesthetic preferences, if one group doesn’t cave, neither group may get the feature.
The problem is there will NEVER be a consensus on this!
Not with that attitude!
I'm being flippant but ES5 took like a decade after ES3 was released to come to an agreement. ES6 took another 6 years. Even optional chaining, which is far simpler than pipeline, took like 4 years. Some features never advance (bind operator), some features go through multiple rounds of revisions (decorators) or last minute changes (globalThis
). When I got started with this, I knew going in this was going to be a multi-year process with no guarantee of success. Designing features for a language used as broadly and in as wide a variety of contexts and styles as JavaScript was never going to be easy. Once features land, there's no going back, so the fact that this takes time is a feature of the process, not a bug.
You've been on this from the start, why can't YOU be the champion here @mAAdhaTTah?
I'm not a TC39 member.
The biggest thing I'm taking away from this idea is maybe we need two operators 🙂
Funny but potentially true :) I picked two operators to illustrate my point, but it might be possible to merge the two operators into one.
It depends on how intuitive the overall feature is in practice. Two operators might seem more expensive than one operator, but it might not be so in terms of mental load if the two operators are similar (e.g. ~>
and ~>>
). >
and >=
are two different operators; +
and +=
are two different operators. I am not saying the situations are exactly the same, just that decisions are usually based on practicality.
@aadamsx
Making up for flawed syntax with linters -- not a great idea @fabiosantoscode
Exactly. This was an argument for the F# proposal, which does less, but is more predictable and less like something people would write lint rules for.
I think @JAForbes makes an excellent point
Anywhere a Fat Arrow function would work, an expression containing the Hack-style argument reference token should work.
We can call them "implicit lamdas" and they can be N-ary and N-order.
#
treats #
as it was bound#
is auto-expanded to fat arrow (#0) => myExpression(#0)
sum([#, #1, #2])
becomes (#0, #1, #2) => sum([#0, #1, #2])
[1,2,3,4].flatMap(arrayOfLength(#).map(##1 * 10 + #))
expands to [1,2,3,4].flatMap((#0) => arrayOfLength(#0).map((##0, ##1) => ##1 * 10 + #0))
which yields [1,2,3,4, 11,12,13,14, 21,22,23,24, 31,32,33,34]
[1,2,3,4].flatMap(arrayOfLength(#).map(##1 * 10 + #).filter((#0) => #0 !== #).filter((#) => # !== #))
#
to (#0) => #0
is semantic, not syntactic; if we stopped here, this first filter would drop the first 4 elements from the end result of the total example expression.# !== #
will always evaluate to false, causing the end result of this total example expression to be []
#
alone is now the identity function. 🤓 [1,2,3,4,5,6,7,8,9]
|> pickEveryN(#, 2)
|> #.filter(# => 0 === # % 4) // `#` in filter lambda explicitly rebinds the token to disambiguate with outer.
// next 4 lines are equivalent, but allow for different nuances in the semantics:
|> shuffle // just a unary function
|> # => shuffle(#) // a unary lambda, which manually binds `#`
|> #0 => shuffle(#0) // a unary lambda, which manually binds `#0`
|> shuffle(#) // interpreted as `(#0) => shuffle(#0)` which is a unary lambda
|> #.map({[##]: `value was ${await dbLookup(##)}`}) // first-order unary lambda of `#0` composing second-order unary lambda of `##0`
|> Promise.all // unary function
|> await // operators are just unary functions
|> forEach(console.log) // TYPE ERROR, assuming no forEach is in scope other than Array.prototype.forEach, which is not called because there is no dot operator.
// Should be `#.forEach(console.log)` or `#.forEach(console.log(##)` or some explicit lambda version of those
The Hack style is just an awful looking compare to F# style.
The current proposal (in which the RHS is implicitly called with the result of the LHS) does not easily support the following features:
In order to better support these features the current proposal introduces special-case syntax and requires the profilgate use of single-argument arrow functions within the pipe.
This proposal modifies the semantics of the pipeline operator so that the RHS is not implicitly called. Instead, a constant lexical binding is created for the LHS and then supplied to the RHS. This is similar to the semantics of Hack's pipe operator.
Runtime Semantics
left
be the result of evaluating PipelineExpression.leftValue
be ? GetValue(left
).oldEnv
be the running execution context's LexicalEnvironment.pipeEnv
be NewDeclarativeEnvironment(oldEnv
).pipeEnvRec
bepipeEnv
's EnvironmentRecord.pipeEnvRec
.CreateImmutableBinding("$", true).pipeEnvRec
.InitializeBinding("$",leftValue
);pipeEnv
.right
be the result of evaluating LogicalORExpression.oldEnv
.right
.Example
Advantages
Disadvantages
Notes
The choice of "$" for the lexical binding name is somewhat arbitrary: it could be any identifier. It should probably be one character and should ideally stand out from other variable names. For these reasons, "$" seems ideal. However, this might result in a conflict for users that want to combine both jQuery and the pipeline operator. Personally, I think it would be a good idea to discourage usage of "$" and "_" as variable names with global meanings. We have modules; we don't need jQuery to be "$" anymore!