Comparison to emacs-async

[seagle0128]

Hi, I am interested in emacs-aio. I am using emacs-async, which is also used in magit. Can you please compare both package? What's the advantage of emacs-aio?

I noticed @tarsius stared aio, maybe he will use it instead of emacs-async in magit if aio is much better.

BTW, aio will be released on melpa or elpa?

[tarsius]

magit doesn't actually use emacs-sync. It depends on it to force package.el to cleanly build it, see https://github.com/jwiegley/emacs-async/blob/master/async-bytecomp.el#L28-L52.

But yes, magit might eventually use aio.

Please add a proper library header and add it to Melpa.

[skeeto]

@seagle0128, on the surface it might seem that async and aio do similar things, but they actually have very different purposes. Because of this, aio is a perfect complement to async, making it easier to use. For example, here's a wrapper around async-start that returns an aio-promise (rather than an async future), making it awaitable:

(defun aio-async-start (func)
  (let* ((promise (aio-promise))
         (callback (lambda (result)
                     (aio-resolve promise (lambda () result)))))
    (prog1 promise
      (async-start func callback))))

That's basically how to turn any callback-based API into an awaitable API. Now, using this function to manage parallel computation in subprocesses without any explicit callbacks and without polling:

(aio-defun hashn (strings n)
  "Iterate SHA1 on each string N times in parallel."
  (let ((promises ()))
    ;; Gather up promises
    (dolist (string strings)
      (let ((func (lambda ()
                    (dotimes (_ n string)
                      (setf string (sha1 string))))))
        (push (aio-async-start func) promises)))
    ;; Await on results
    (cl-loop for promise in (nreverse promises)
             for string in strings
             collect (cons (aio-await promise) string) into results
             finally return (cl-sort results #'string< :key #'car))))

It seems async doesn't limit the number of processes to run at a time, so this could get out of hand when there are lots of strings. The aio README has an example of using a semaphore to limit the number of promises in-flight at a time, which would essentially retrofit a work queue atop async.

@tarsius Will do, though at the moment I'm still figuring out what all the interfaces should look like, so the API keeps changing. If you feel you're going to use this someday, feel free to suggest API changes since I don't care about breaking anything right now. I've been writing lots of little programs and servers using aio to find the rough spots.

[tarsius]

I haven't had a chance to try this out yet, it probably will be a while until I do.

I agree you should delay adding this to Melpa. It might even make sense to add it to GNU Elpa eventually, but then you would have to change the license.

[seagle0128]

Thank you so much for the details above! I am fine to close it now.

[kaiwk]

Hi, @skeeto, sorry for bothering ; )

I'm very interested in this package, since async/await and future/promise are very popular now. I also find a package called emacs-deferred.

Since emacs-aio is single threaded now, it may not suitable for CPU bound task, but in emacs, in most case, we have IO bound tasks, such as networking and read/write files.

async/await is just an abstraction of callback, so this package may use emacs multi threads in the future. By now, when executing some very time cost task, I think it'll still block Emacs.

Besides, I am curious about the implementation of run-at-time, it looks like setTimeout in JavaScript. Does that mean Emacs(elisp runtime) have a event loop like Node.js?

[skeeto]

Emacs can only run one thread at a time, but aio is otherwise not really single threaded itself. If Emacs' threads were to become useful in a future Emacs release, you could create a thread to compute something but await on it in a function "running" on the main thread. You still even do this now. Here's an example:

https://gist.github.com/skeeto/6f0e148c3eb4e40267a49c2059385b5d

In batch mode it just prints out result. In non-batch mode, the UI remains responsive (due to those thread-yield calls) while the threads do their work. The main thread receives the result via aio promise and prints it into the current buffer. Caveat: there's no synchronization on the promise object, so this only works as long as Emacs threads cannot be pre-empted (which is currently the case).

You're correct about run-at-time. It's essentially the same as setTimeout, and using run-at-time with a zero time is really just scheduling an event on the next turn of the Emacs event loop.

[Silex]

@tarsius: if you are interested, look at https://github.com/Silex/docker.el/blob/master/docker-volume.el#L88-L92 and see how clean it looks to call two async processes and wait on them then process results of both.

The make-process version would be a nesting nightmare.

[tarsius]

I still plan to investigate the use of this library, so thanks for the example!