Investigate Promises/A+ compatibility w/a Monad system

[briancavalier]

I see a lot of potential value for the JS community in having Promises/A+ promises be compatible with a monad system. I can relate to @raganwald's thoughts in the now infamous #94.

As has been mentioned several times in that thread, there is the very real, practical concern of breaking the web. Promises/A+ has had the messy task of dealing with the current state of the world and interoperating with other widely used non-Promises/A+ things, like jQuery Deferred, as well as Promises/A implementations, like Dojo Deferred. I feel that it has been extremely successful in that regard. It's impossible to say at this point whether a clean break would have been the right thing to do six months ago when the first draft of Promises/A+ appeared in a gist.

We are where we are. Promises/A+ exists. It is compatible with lots of things, past and present. People have used and are using it to write software that provides real benefit to real users.

If it's possible to devise a way for these two elegant and powerful concepts to work together without breaking the web, I am in strong support of that.

Therefore, I'd like to suggest a course of action.

Form a small, combined team of folks from Promises/A+ and Fantasy Land to figure out a way to make this work while maintaining Promises/A+'s current level of compatibility, and prove it with working code. One possible approach this team could take is to choose a very simple Promises/A+ impl, like avow (or whatever this team decides is best), and make it work in fantasy land while still passing the P/A+ 1.1 test suite.

[unscriptable]

This sounds like a great idea to me.

and prove it with working code

and tests?

choose a very simple Promises/A+ impl

I agree. Too much cruft in the library will just distract the team.

[briancavalier]

I am encouraged by the constructive/exploratory tone of the discussion by @juandopazo and @pufuwozu over in promises-aplus/resolvers-spec#24

[Twisol]

I'm certainly interested in this (even though I'm not technically part of either team).

[wizardwerdna]

Imagine what wonders may come.

[briancavalier]

In the interest of furthering the exploration, I created a branch of avow that implements of(x). It doesn't change the behavior of then().

[puffnfresh]

@briancavalier that looks absolutely brilliant.

[briancavalier]

Thanks, and thanks to @Raynos for the nudge in briancavalier/avow#2. @pufuwozu could fantasy-land provide some code snippets that we can use to demonstrate (as in show something actually running in node or wherever) how abstract operations can be performed on promises and other data types with the presence of this compliant of(x)? I think that kind of concrete demonstration would go a long way.

In a somewhat similar vein, the Promises/A+ test suite has proven to be a huge win for learnability and demonstration purposes, so I think the same would hold true here--it may make it easier for people to learn from our experimentation.

[rtfeldman]

Props to @briancavalier for bringing this topic into a very positive place. This direction seems awesome! Really looking forward to seeing where this ends up.

[puffnfresh]

@briancavalier yes and yes. More examples, implementations and test suites will be provided. Definitely want to make it easier to be compliant :smile:

[juandopazo]

After dedicating some thought to all of this, I reached a conclusion. In order for promises to be monadic two things must be true:

1: There has to be a way to create a promise for a promise

2. then must not recursively assimilate promises returned from the success callback

Unfortunately, version 1.1 of the A+ spec breaks the second requirement.

YUI still hasn't adopted 1.1 so it can be used to make monadic promises. Example: http://jsbin.com/ewegat/1/edit/

[Raynos]

@juandopazo yikes! Why is it recursive? :(

[juandopazo]

@Raynos I'd rather we keep this thread/issue low noise and focused on findings and experiments. You can probably look up the pull request that added the assimilation procedure and read the reasoning there

[juandopazo]

I updated the examples so that it's clear what's going on:

• YUI, A+ 1.0 compatible: http://jsbin.com/ewegat/6/edit/
• when.js: A+ 1.1 compatible: http://jsbin.com/awivov/1/edit/

[Raynos]

@juandopazo from reading the the specification it appears it's only recursive for thenables (2.3.1) and not recursive for promises (1.2). this would not cause an issue for monads as the monadic then should return a promise and not a thenable.

[ForbesLindesay]

It still unwraps one layer each time, and the current thinking for promise creation is that it should be done via a "resolve" method which also unwraps one layer.

[Raynos]

unwrapping one layer each time is the correct behaviour for monadic then

[ForbesLindesay]

Hmm, but "Resolve" also unwraps a layer, so there's not any method of creating a promise for a promise, since any layer you add is instantly removed.

[Raynos]

@ForbesLindesay that requires a new of function.

var p2 = p.then(function (pValue) { return Promise.of(Promise.of(pValue)) })

p2 should be a promise that contains a promise. i.e. then returned and saw it was a promise and said that the value of p2 is the value of the promise which is a promise.

[ForbesLindesay]

Hmm, the issue is that I still struggle to find a compelling behavior to support creation of promises for promises.

[wizardwerdna]

I don't see anything in the spec that precludes creation of a fulfilled promise whose value is a promise. It defines a promise as comprising three states, the fulfilled state having an unspecified value.

The only limitations on these states and values is that, once fulfilled, neither the state nor the value can be changed again. Nothing in the specification precludes the existence of a fulfill or resolve function that verbatim feeds its values to the promise, without reference to a "resolution process" or otherwise.

I'm not certain as to their utility, but I believe those operations can be provided, consistent with the spec. The spec does specifically provide for the operation of then, however, complete with a resolution process.

On Fri, Apr 12, 2013 at 6:44 PM, Forbes Lindesay notifications@github.comwrote:

Hmm, the issue is that I still struggle to find a compelling behavior to support creation of promises for promises.

— Reply to this email directly or view it on GitHubhttps://github.com/promises-aplus/promises-spec/issues/97#issuecomment-16325543 .

[ForbesLindesay]

It doesn't preclude the creation of a promise for a promise at the moment. The resolvers spec has a current draft that does not include any method that would allow you to create a promise for a promise though. This doesn't mean you can't do so, but if you want to specify an API that allows you do do so and get it published under the promises-aplus namespace, you'll need to make a case for why it's important.

[Raynos]

@ForbesLindesay the only argument for a promise of a promise is to make the stacked async behavior obvouis.

p.then(function (x) { return x * 2 }) will fulfill once p fulfills.

p.then(function (x) { return ajax(x) }) will fulfill once p and ajax fulfills.

The fact that it's not obvouis that you have a flattened promise of a promise means it may be too easy to write waterfalled dependencies on multiple async calls.

With callbacks the fact that you have a 5 levels of identation makes you think "maybe I should run these async tasks in parallel". With the automatic flattening there is no encouragement to run asynchronous tasks in parallel.

[erights]

@Raynos I don't understand this. Not objecting; at least not yet. I just don't understand. Perhaps another example? Or perhaps explain what lost parallelism you are concerned about in this example?

[puffnfresh]

@Raynos that is not the only argument. A more important argument is that it breaks laws. I will come up with an example soon.

[erights]

@pufuwozu FWIW, I got the "it breaks laws" issue. Though of course more clarifying examples are always welcome. Thanks.

[Raynos]

@erights the fact that return x and return ajax(x) are both valid in .then() means it's not obvouis whether a chain of 10 .then() calls will fulfill synchronously once the first one is done or will make 11 asynchronous calls all in series.

Having some kind of syntax to make it more obvouis that you have a Promise<Promise<X>> instead of Promise<X> will make it more obvouis that the former is two asynchronous actions in serial and thus is generally slower by amdahl's law ("The speed up of a parallel program is bottlenecked by the sequential subset of the program").

Promise<Promise<X>> makes it obvious that there is a sequential part of the program.

In general where possible converting (Promise<X>, Promise<Y>) into Promise<(X, Y)> instead of Promise<Promise<Y>> is preferred because it allows the two promises to fulfill in parallel and is generally not sequential.

TL:DR; Making it easy to write sequential programs where it can just as well be written in parallel leads to slower programs.

[Raynos]

Furthermore when compared to callbacks. Writing a sequential program looks really ugly due to nested callbacks. This is a visual cue for me as a program writer to fix my code and parallelize it.

Promise<Promise<Promise<Promise<X>>>> is the exact same parallel in promise world. It's something that looks very ugly and is a visual cue for a program writer to fix his code.

Having then() flatten it out means that a chaining of multiple then() statements is either ugly or not depending on whether each then() returns a promise which can be very hard to visually identify when you use functions that may return values or promises.

Obvouisly if you choose to have a long chaining of explicit map() (return a value synchrnously) and flatMap() (return a promise synchronously) it would again be easy for a programmer to identify large sections of unnecessary serialized code.

TL:DR; Making the type Promise<Promise<Promise<X>>> explicit instead of implicitly flattening it out by default allows for you to visually identify functions which have an unnecessary amount of serialized flow going on.

[Twisol]

I made a similar argument as @Raynos' over here.

[Raynos]

A concrete example of this problem that I've personally had with promises recently is porting @medikoo promise example from his deferred project ( https://github.com/medikoo/deferred#promises-approach ) to something promise like that doesn't have automatic flattening ( https://gist.github.com/Raynos/b8bf27d5d05811858bfc#file-better-chaining-js )

Now one thing to note through porting this if you take a look at my revision history ( https://gist.github.com/Raynos/b8bf27d5d05811858bfc/revisions ) I actually added flatten in later once I realized "wait a second this doesn't work. I'm trying to map continuables instead of values". my first reaction to this was "i shouldnt need flatten let me fix my code". My second reaction was "oh I have serialized dependencies. Great :( ok I will flatten them". My third reaction was "Medikoo's example really doesn't make the serialized dependencies obvouis and his map function is black magic".

TL:DR; I care about reducing the amount of serialization in my program. I need the serialization to be explicit otherwise it's not trivial to identify "ugly serial code" and casually refactor it. Making something I don't like ugly (Promise<Promise<Promise<X>>> I don't like it. It's ugly) allows for a really healthy casual refactoring attitude towards it. Making it hidden or implicit means it's really hard to identify and fix and won't be fixed by default or by habit. I don't write nested callbacks by habit, I've acquired an actual habit to make this anti pattern just not happen by default.

[ForbesLindesay]

Generally speaking I find that I use chains of .then calls for operations that have to operate in serial because each one depends on the result of the previous one. There's plenty of those to make it a case worth optimizing for. Also, the promise semantics guarantee that a call to then is always asynchronous, no matter what, so you're ambiguity seems imaginary.

I really can't help but think we should be working off issues people have actually encountered. It should be that you lead with "Can we have nested promises because of this really important use case." Instead people are dreaming up increasingly obscure and abstract reasons why someone might need this feature in the future.

[puffnfresh]

I really can't help but think we should be working off issues people have actually encountered.

@ForbesLindesay you're suggesting a broken of. People haven't hit the problems that will occur when that happens. I will have to contrive one.

This function doesn't exist yet and you've asked me to prove that things will break if you break the function. You haven't proven why it's useful to not allow it, either.

This whole situation is absolutely ridiculous but I'll play along - it won't be too hard to show you something that will break, just a little time consuming.

[briancavalier]

This thread has gotten a bit noisy. While some level of debate will be necessary, let's all please remember that the purpose of this issue is to find a way to work together through concrete experimentation.

[briancavalier]

@pufuwozu Has there been any progress toward providing example code for abstract operations into which we can plug avow's of implementation?

Would it help pave the way for more experimentation if I also provide another branch of avow that includes a version of then() that only flattens one level, a la flatMap? Then we would have 2 branches, both with a working of (as far as I know, but please let me know if it does not do what you are expecting it to do) with which to experiment.

[puffnfresh]

@briancavalier here's some stuff that works for abstract structures of the Fantasy Land specification:

https://github.com/pufuwozu/fantasy-sorcery/blob/master/index.js

A chain method that only assimilates a single level sounds awesome.

[medikoo]

@Raynos deferred.map is analogous to Array#map, it just returns a promise that resolves when all array values are resolved, I think it's pretty straightforward, where's black magic? I don't get :)

[briancavalier]

Ok, I've added an implementation of chain to the fantasy-land avow branch. I don't know if it's correct. I also added Promise.of so that p.constructor.of is available.

I grabbed @pufuwozu's fantasy-sorcery code and wrote a simple test using liftA2, mainly as a point of discussion as to whether chain is doing what it needs to or not. It also shows the difference between using avow.of and avow.from.

Here's the output of liftA2-test.js:

123 456
123 456
{ then: [Function: then] } { then: [Function: then] }
123 456

Is that correct? If so, then it'd be great to write some more substantial tests. If it's not correct, then I'll need some help understanding exactly what chain is supposed to do, and preferably some code that helps me verify. Thanks!

EDIT: Here's a direct link to the chain and of functions

[puffnfresh]

@briancavalier perfect. I also added this one:

fl.lift2(function(a, b) { return a + b; }, avow.of(1), avow.of(2)).chain(function(x) {
    console.log(x);
    return avow.of(x + 2);
});

Correctly prints out "3" and gives a new promise of "5". Very awesome.

[junosuarez]

Is there a functional notion of a variadic liftN which would apply to any number of same-class applicatives? Or am I reading this wrong, and that wouldn't make sense?

[Twisol]

@jden: Yep. It's called ap, often used as an infix operator in Haskell as <*>. Because Haskell's version relies on partial function application, we wouldn't have the same kind of ap, but it's conceptually the very same thing. You're right that it would be variadic.

A Haskell example: [(1+), (1-)] <*> [1, 2, 3] produces [2, 3, 4, 0, -1, -2]. You're taking a function within a structure (a List here) and applying it to an argument in the same kind of structure. This particular applicative produces the cartesian product of functions and parameters - that is, it applies every function to every parameter.

(For clarity: (1+) is the function that adds its parameter to 1. (1-) is the function that subtracts its parameter from 1.)

The functions here only take one parameter, but if you have functions that take multiple, you can just use <*> a second time: [(+), (-)] <*> [1] <*> [1, 2, 3]. And so on. But again, in Javascript ap would be variadic, so you'd just ap once but pass an arbitrary number of parameters.

[puffnfresh]

@jden we should be able to write a liftN - I didn't think it was worth my effort at the time. Pull request welcome! :smile:

[ForbesLindesay]

@pufuwozu @jden @Twisol can you move this discussion to the Fantasy Land repo. It's not really related to Promises-A+

If you are interested, we already have something with the functionality of liftN and we call it spread:

Q.spread([in1, in2, in3], function (arg1, arg2, arg3) {
});

Which is just shorthand for:

Q.all([in1, in2, in3]).spread(function (arg1, arg2, arg3) {
});

Both methods support completely variable numbers of arguments.

[briancavalier]

I feel good about the results of this experiment. I think we proved a few important things:

1. It is possible to implement of(x)
2. It is possible to implement chain (although I personally would like to see more validation)
3. At least initially, It seems possible to implement both while still passing the (as-yet-unreleased) P/A+ 1.1 test suite

The goal of this thread, as originally stated:

figure out a way to make this work while maintaining Promises/A+'s current level of compatibility, and prove it with working code ... choose a very simple Promises/A+ impl, like avow (or whatever this team decides is best), and make it work in fantasy land while still passing the P/A+ 1.1 test suite

I feel we've accomplished this, so I'm closing this thread. Thanks to everyone who helped.

Please direct any energy toward #101, and please be especially cognizant of the process that @ForbesLindesay has established there for continuing to move forward. Please also direct any discussions not related to Promises/A+ in some way to other, appropriate places, such as Fantasy Land github issues.