Easier way to create transducers

[asaf-romano]

See issue #1656 for background.

My experience with transducers so far shows that they are worthless if the accumulator transducer does not avoid creating intermediate objects by itself. This means that something like transduce(compose(...), mergeWith(...), {}) won't perform well on a large data sets. Problem is, the transducers protocol, while very powerful, asks for quite an ugly object, that is tiresome to write.

It would thus be nice to have something like Java's Collector.of, which allows initiating Java's analogous of accumulator-transducer, given its building up blocks. See: https://docs.oracle.com/javase/8/docs/api/java/util/stream/Collector.html#of-java.util.function.Supplier-java.util.function.BiConsumer-java.util.function.BinaryOperator-java.util.function.Function-java.util.stream.Collector.Characteristics...-

(Ignore combiner and Characteristics)

[CrossEye]

I'm trying to make sense of what that would mean in JS. Do you have a sense of what one might supply to a Ramda function to use something like this? There doesn't seem to be a simple translation of the Java types. Supplier :: Unit -> a ? BiConsumer :: b -> a -> Unit ? For side effects?, but with the more sensible BinaryOperator :: a -> a -> a that you want us to ignore. I'm just not getting it.

[asaf-romano]

It'd be as simple as thatNewFunctionName(init, step, result). So, for instance: thatNewFunctionName(Array, (acc, x) => { acc.push(x); return acc }, uniq)

This isn't so different from the current object you pass in, but there's a better chance people will bother.

[CrossEye]

How much of this do you think we could embed in internal calls? I really don't like the idea of exposing or asking users to supply mutable structures. I don't at all mind using them internally.

Or perhaps even if we do offer the general API, could we create a simpler gloss for some types, especially lists?

[asaf-romano]

At least in Java (sorry, don't really know Clojure) the whole idea behind collectors is that you sandbox the mutations the way I did in the PR today, so while the mutations are technically mutable, they have no effect on outside-usage of the (finalized) data structure. Now, it's a little hard to demonstrate that in plain JS without in-your-face array.push kind of mutations, because there's no "builder"* concept that emphasizes the fact the object is intermediate by its nature.

Immutable.js has this asImmutable/asMutable concept, which gets very close, and I suppose other libraries provide similar utilities. But I doubt we'll ever use any of them in sample code.

---

• Note that Java builders very often do not provide means to get the data that is already set into them, which makes the distinction between builders to immutable data structures even clearer. I wonder if we can somehow provide a similar building block for arrays and objects.

[kevinbeaty]

To expand on this, you can think of Java's Collector <=> JS transducers:

• supplier <=> @@transducer/init
• accumulator <=> @@transducer/step (kind of, except step returns new accumulator)
• finisher <=> @@transducer/result

So you could define, e.g., Collector.of <=> R.transformer (kind of)

R.transformer = R.curry(function(init, result, step){
  return { 
     '@@transducer/init': init, 
     '@@transducer/step': step, 
     '@@transducer/result': result                                                     
  };
}); 

How would we use it? Recall a basic transducer:

var transducer = R.map(R.add(1));
var x = R.into([], transducer [1,2,3]);                                              
// [2, 3, 4]

Also recall that into will accept a transformer:

var xf = R.transformer(Array, R.identity, R.flip(R.append));
var x = R.into(xf, transducer, [2,3,4])
// [3, 4, 5]

You could curry the transformer to work with arrays:

var transformArray = R.transformer(Array, R.identity);
var x = R.into(transformArray(R.flip(R.append)), transducer, [3,4,5]);
// [4, 5, 6] 

var x = R.into(transformArray(R.flip(R.prepend)), transducer, [3,4,5]);
// [6, 5, 4]  

Or define a transformer (using a "mutable reduction") for Immutable.js

var transformList = R.transformer(
    () => Immutable.List().asMutable(),
    (acc) => acc.asImmutable(), 
    (acc, item) => acc.push(item));

var x = R.into(transformList, transducer, Immutable.List.of(5, 6, 7));
// List [ 6, 7, 8 ]
// NOTE: the result is an Immutable List but the reduction was mutable

You could also curry into to create a reusable intoList:

var intoList = R.into(transformList);

var x = intoList(transducer, Immutable.List.of(6, 7, 8)); 
// List [ 7, 8, 9 ]

In many cases, the result (finisher) is identity, so it may make sense to flip result and init and common transformers can use R.transformer(R.identity).

R.transformer = R.curry((result, init, step) => ...

Note that Java also provides an overridden Collector.of that sets the "finisher" (result) to identity, so it was common enough to include in their API as well.

[CrossEye]

Thank you both for the interesting information. It looks like we have some room for additional optimizations here.