Scope creep

[masaeedu]

I may be misunderstanding the proposal, but I don't understand why you need the ability to do [] :> f or Promise.resolve(x) :> f or the other @@lift operations defined in there. I guess it's a good idea to use symbols to keep implementation flexible, but I think the actual proposal should be restricted to function composition to keep things simple and avoid mental overhead.

The more things the operator can do the more you need to squint at every member in a pipeline to understand the exact semantics.

As I said, I may be misunderstanding the proposal, happy to be corrected if so.

[dead-claudia]

@masaeedu From a category-theoretic perspective, @@lift is analogous to a functor's map. And as mentioned in this section, if you squint a little, function composition and promise chaining look very similar.

I won't generalize that operator any further than this unless I'm presented with math that tells me that a functor isn't the top of that hierarchy anymore. (And trust me when I say that'd be incredible news all over. Haskell/etc. fans would be groveling out the ears over that. Or in other words, it probably won't happen for quite a while.)

As for why I chose to broaden the scope: I was looking to address not only function composition, but also the vast number of reactive operators and other collection-oriented operators that 1. shouldn't need to be so dependent on observables (_.uniq works for both arrays and observables), and 2. shouldn't be so hard to define for the general case. What the two elaborated variants do:

• :> is basically "mapping" over a type, in a very loose meaning of the term. It could mean "mapping over an argument" (e.g. function composition) or "mapping over an entry" (e.g. .map for arrays). This could be useful for just defining simple transforms in a pipeline as well. It addresses three issues at once:

  • Function composition, or lack thereof. (This is easy to tell from context unless your functions are named like collections, which is rarely a good idea to start.)
  • Lack of a simple .map for generators and similar, which has inconvenienced several people already.
  • A very disjoint syntax for async transformation. (Async iterators, observables, and promises are nothing alike, but they can all three be "mapped" over in a sense.)

• >:> (the expansion I went into detail on) is basically "map" + "flatten", but combined into a single operation. This enables most of the reactive operators to be defined more generally, without even targeting observables/streams in particular.

  • It'd be very valuable for something like Lodash's _.uniq to be properly defined across not only normal arrays and Lodash wrapper types, but also Rx observables.

---

Function composition is easy, but I'm trying to see how much of the rest of this I can cover with as little surface area I can, especially with as little new syntax as practically possible, and without succumbing to the serious scope creep that one has. To be more precise, here's what those two address, out of that list:

• [ ] Core Proposal (Basic pipeline syntax - this is orthogonal to my proposal.)
• [ ] Additional Feature BP (Block pipeline bodies - also orthogonal to my proposal.)
• [ ] Additional Feature PP (Pipelines within such blocks - also orthogonal to my proposal.)
• [x] Additional Feature PF (Pipeline functions - I solve this by allowing composition.)
• [ ] Additional Feature TS (Pipeline try statements - also orthogonal to my proposal.)
• [ ] Additional Feature NP (N-ary pipelines - one of my open questions.)
• [x] Additional Feature FS (Pipeline for / for await statements - I solve this by defining a more general map/flatMap.)

I personally disagree about some of their decisions (specifically that of making pipelines not simple binary operators), but that's a conversation I've found little success in even attempting to get anywhere with.

[masaeedu]

@isiahmeadows The . composition operator that you use day to day in Haskell isn't an alias for fmap, even though fmap for functions is composition. There's lots of ways to implement functors, and not all of them involve putting things on the prototype of obejcts, so I still think this is a serious case of scope creep.

Regarding the pipeline operator, I similarly wish that it would just be reverse partial application rather than a bajillion other overloaded things, but I guess that's a discussion for a different repo.

[dead-claudia]

@masaeedu

The . composition operator that you use day to day in Haskell isn't an alias for fmap, even though fmap for functions is composition.

I know it's not. More to the point, composition for functions in many more recent functional languages is typically defined for Semigroupoid (usual subtype of Category), in which function types are an instance of. (This is the case for PureScript, which defines only >>>/<<< on that type class for composition.) To expand on this further, functions are also a type of Arrow, a subtype of Category, in that there 1. exists an identity, and 2. exists a way to "split" them.

But equivalent ≠ alias, and mathematically, equivalence works well enough. (It's an unusual application, and in an object-oriented language, it works well enough.)

As an item of note, Fantasy Land is pursuing strong profunctors (basically, profunctors with the ability to "split") over arrows. Profunctor is a subtype of Functor, and the proper definition for those methods for functions are this:

const map = (f, g) => x => g(f(x))
const promap = (f, a, b) => x => b(f(a(x)))
const first = f => ([a, b]) => [f(a), b]

There's lots of ways to implement functors, and not all of them involve putting things on the prototype of obejcts, so I still think this is a serious case of scope creep.

I'm aware. But here's my question (it's two-fold):

1. How would you implement functors without using magic properties?
2. How would existing libraries migrate to this method?

I know it's possible to implement functors using object factories, much like how OCaml uses module functors to emulate type classes and how the competing Static Land (Fantasy Land offshoot) models its types. However, few libraries would be able to migrate well to it, and several people/groups/companies would find it difficult to move their more object-oriented code bases to it, especially if they're heavy users of Lodash, RxJS, and the like.

And to be clear, what's listed in the "Possible expansions" is about as far as I plan to draw the line. (I'm not planning on redesigning JS.)

Regarding the pipeline operator, I similarly wish that it would just be reverse partial application rather than a bajillion other overloaded things, but I guess that's a discussion for a different repo.

Yeah...I've already criticized it there elsewhere briefly, but for whatever reason, any future discussion about it got shut down pretty swiftly (which didn't come across as particularly civil for an issue where it wasn't even topical). This is also why I kept it out of this proposal - it was easier to focus on the content rather than the semantic noise.

[masaeedu]

@isiahmeadows The different hierarchy in PureScript is interesting, but ultimately I'd be very surprised if they don't special case function composition to a simple monomorphic compose = f => g => a => f(g(a)) when transpiling, rather than doing the magic polymorphic dispatch every time.

Regarding how I would implement functors, I'd implement them like this:

// arr.js
export const map = f => xs => xs.map(f)

// fn.js
export const map = f => g => a => f(g(a))

// etc.

Whether this is a good approach remains to be seen, but I'd prefer to have this discussion at the library level rather than prematurely baking all this stuff into the language. It's not even that I dislike the approach to structuring functors you have; it's just that I don't think it's a sensible part of this proposal, and puts the cart way before the horse.

Ideally, we'd have one proposal for user-specified infix operators (or infix application of existing identifiers), and we'd be able to deal with all the churn in all these different proposals at the library level rather than prematurely baking everything into the language. The language committee seems to be 👎 on that idea based on previous discussions, so the next best thing is to make each incremental operator do as little as possible, as uncontroversially as possible, while leaving yourself open for future improvements (e.g. an fmap operator that dispatches to composition for functions).

[dead-claudia]

@masaeedu Mine is pretty minimal to start, and I've been very careful to not complicate it more:

// x :> f
function pipe(x, f) {
    if (typeof f !== "function") throw new TypeError()
    return x[Symbol.lift](f)
}

The two eventual additions (one inline in the README, the other in #6) I have are a bit more involved:

• N-ary lifting (#6):

  // [a, b] :> ...liftFunc, optimized
  a[Symbol.lift2](b, liftFunc)
  
  // [a, b, c] :> ...liftFunc, optimized
  a
  [Symbol.lift2](b, (a, b) => [a, b])
  [Symbol.lift2](c, ([a, b], c) => liftFunc(a, b, c))
  
  // arr :> ...liftFunc, requires runtime call
  function naryHelper(iter, func) {
      const array = Array.from(iter)
      const len = array.length - 1
      if (len < 1) throw new RangeError("array must be at least of length 2")
      if (len === 2) return array[0][Symbol.lift2](array[1], func)
      let acc = array[0][Symbol.lift2](array[1], (a, b) => [a, b])
      for (let i = 2; i < len; i++) acc = acc[Symbol.lift2](array[i], (as, b) => [...as, b])
      return acc[Symbol.lift2](array[len], (as, b) => func(...as, b))
  }

• Pipeline collection manipulation: This one is a little more involved, but that's because it really covers three features at once (.flatMap(f) + .takeWhile(f) + .from([...])) in a way that I'd like to allow all three in one pass.

  • The sync version is only about 25% larger than the lift2 one above:

    function invokeChainSync(coll, func) {
      if (typeof func !== "function") throw new TypeError()
      return coll[Symbol.chain]((...xs) => {
          const f = func
          if (f == null) throw new ReferenceError()
          const result = f(...xs)
          if (result == null) { func = void 0; return }
          if (Array.isArray(result)) return result
          if (typeof result[Symbol.chain] === "function") return result
          if (typeof result[Symbol.iterator] === "function") return Array.from(result)
          throw new TypeError()
      })
    }

  • The async version has edge cases it needs to track. Ideally, it'd be equivalent to the sync version and about 50% of the size, but there's a few things it needs to handle:

    1. Inner callbacks are executed in parallel and all need awaited simultaneously.
    2. A later callback might force a break before a previous one returns a nested value.
    3. The outer coll[Symbol.chain](func) call might resolve before all its active callbacks resolve.

[dead-claudia]

Ideally, we'd have one proposal for user-specified infix operators (or infix application of existing identifiers), and we'd be able to deal with all the churn in all these different proposals at the library level rather than prematurely baking everything into the language. The language committee seems to be 👎 on that idea based on previous discussions, so the next best thing is to make each incremental operator do as little as possible, as uncontroversially as possible, while leaving yourself open for future improvements (e.g. an fmap operator that dispatches to composition for functions).

With custom operators, you have three very major complication points, which make it hard to add into a language without designing the language for it in the first place:

1. Parsing. Custom operators inherently make the language context-sensitive, much more than even JS's ASI algorithm. They also require that you specify associativity at declaration time and also either 1. specify precedence/associativity at import time or 2. parse imports just to parse a file. It also introduces several potential new ambiguities, especially if you allow identifier operators (OCaml supports only symbol-based operators).
2. Runtime. Custom operators require that you replace many common native operations (like addition/concatenation) with potentially dynamic calls. You can currently inline the implementation of every single operator unconditionally and still follow the spec. Engines use this flexibility when optimizing hot code paths to just inline what they need, and this would violate many of those assumptiohns.
3. Compatibility. asm.js requires that x | 0 yields a signed integer, that x >>> 0 yields an unsigned one, and that +x yields a double. It also requires that all primitive operations on builtins do not change. It was specifically for asm.js compatibility why +1n, 1n >>> 0, and 1n | 0 throw, and violating those assumptions quickly leads to major issues.

[dead-claudia]

(They are not against operator overloading AFAICT, within certain constraints.)

[masaeedu]

@isiahmeadows

Parsing

All of these difficulties only arise if you do not have a dedicated delimiter for infix application. Moreover, all these difficulties have to be dealt with anyway for all the new operators being proposed, except that right now they're being dealt with across language proposal repos and are immutable for all users of JS once you stick them in the language

Runtime. Custom operators require that you replace many common native operations ...

Custom operators do not require this, although you're free to do it if you want. You can still have reserved operators in the language that are not available for user-redefinition. Conflict is impossible if you have delimited infix application.

Compatibility

I don't understand how this is relevant. See above.

[dead-claudia]

@masaeedu I'd suggest talking to an actual TC39 rep about custom operators, since they would have a better idea what exactly what issues would likely block/inhibit custom operators.

[dead-claudia]

BTW, I've done a pretty large update to both the formatting and proposal content, including a ton of just explanatory content. Specifically, about the proposal itself:

• I reified the main "possible addition", the chaining, and I simplified it some.
• I reified and refined the feature in #6, electing to do a function rather than adding syntax (if syntax starts looking and acting too much like a function call, why not just make it a function call?).
• I cleared up a few edge cases with the basic lifting method, and made it consistent with the rest (for each sync variant, there's also an async variant with near-equal complexity).
• I touched on and addressed a few other edge cases throughout the proposal.
• I also touched on a few of the rough edges and smoothed some of them out.

The major edits beyond above added practically nothing besides prose and clarity, however. The "possible additions" section now only have two entries:

1. Object.box(value) - A simple built-in option-like abstraction (really, a type-safe nullable) for use with the pipeline. This is 100% polyfillable as a simple, trivial builtin, but it's here because engines could just obliterate the whole abstraction at code gen time if they put the right ICs in for type feedback.
2. Cancellation integration - This is highly dependent on the fate of the cancellation proposal, and is just acknowledging a tangentally related concern.

Most everything else is either already factored in or is something I'd likely reject as out of scope. My end goal is to bring the useful bits of Fantasy Land and other similar specs/utilities (like the old observable spec, Promises/A+, the ES iteration protocol), and create something that's easy to use and easy to implement.

---

As for why I let it evolve past function composition? That particular concept is narrow enough it's like putting a bandaid where we already have people making oversized ones for us (Lodash's _.flow, Ramda's R.compose, etc.), and I wasn't fully convinced the extra syntax was truly worth it. It just felt too small to merit wasting syntax for that one little thing.

Expanding it to encompass collections and pseudo-collections more broadly also enabled me to take a second stab at this problem, in a much simpler, less intrusive way. (90% of that functionality is wrapped into Object.combine and the >:> operator.) So now, I feel that the two operators are really carrying their weight.

In case you're curious what the proposal entails, I have basically a cliffs' notes section at the top, if you just want it at a glance. (I added that since the extensive prose kind of obscures the not-so-large scale of the core of the proposal. 90% of the code implementing it would be over the library additions, not the core syntax.)

[babakness]

Is there a way to try this in Babel right now? Would love to try it out.

[dead-claudia]

@babakness Not currently, but I'm not a heavy Babel user (I'd more likely just fork Acorn or Esprima to try it). If you feel strongly enough about it, please file a separate issue, since this one is about completely different concerns.