a promise which returns a promise gets stuck in pending

[vttrifonov]

This works

f <- future({
    as.promise(future({Sys.sleep(2); 2}))
}) %...>% cat()

and prints 2 after about 2 secs (as expected).

This does not work

f <- future({
   future({Sys.sleep(2); 2}) %...>% { . + 1 }
}) %...>% cat()

the returned promise is stuck in pending. I was expecting/hoping to see 3 printed after about 2 secs.

I was playing around with promises and ended up with a promise which used a piped promise (more or less like the above) in its calculation, but it never resolves, making the above pattern useless. Is this intentional? I am new to this, so perhaps there are fundamental problems with promises returning promises The first example makes me think it is not so much 'promises returning promises' (which seems to be fine in the simple example), but 'promises returning pipes' which is kind of strange...

[vttrifonov]

This also gets stuck in pending

f <- future({
     promise_resolve(2) %...>% {. + 1}
 }) %...>% cat()

So it does not seem to be a problem with having a future within the future (which was my first guess). I little playing around shows that the issue is with calling later::later from within a future. For example,

value(future({cat("hi\n")}))

prints hi and returns NULL, while

value(future({later(function() cat("hi\n"))}))

just returns NULL and never prints hi. Presumably, in this case, the function passed to later is not executed. My guess is that in my latest example with promise_resolve, the call to handleFulfill on line 171 in promise.R is never executed and so the pipe promise is left in a pending state.

[jcheng5]

I'm on vacation so I won't be investigating this immediately, but my guess is that this is related to future topology (https://cran.r-project.org/web/packages/future/vignettes/future-3-topologies.html). If a future is run with a sequential plan() (which is the default for nested futures) then it will not ever resolve. I would be surprised if most use cases for nested futures could not be rewritten to unnest the futures, though...? (Nesting promises, on the other hand, is natural and often necessary)

[vttrifonov]

No problem. Enjoy your vacation! I am gonna record my thoughts and you can come back to this on your time. I do not have much rush.

I think my second example (with the promise_resolve where there is no nested futures) shows that the problem it is not with the nested futures (and any possible issues with topologies which was also my guess about the problem). I experimented some more with the later package and this is what I can do to make this work

   f <- future({
       future({Sys.sleep(2); 2}) %...>% { . + 1 } %T>% { run_now() }
   }) %...>% cat()

prints 3 after about 2 secs as I was expecting initially. The run_now at the end of the pipe flushes the hanging later callbacks and lets the promise finish its state logic. It is not pretty, but it might patch me over until you have time to comment more.