deferred.el
provides facilities to manage asynchronous tasks.
The API and implementations were translated from JSDeferred (by cho45) and Mochikit.Async (by Bob Ippolito) in JavaScript.
(note the README for concurrent
is here in the same repo)
You can install deferred.el from MELPA by package.el.
You can find following sample codes in deferred-sample.el
.
Executing eval-last-sexp
(C-x C-e), you can try those codes.
This is a basic deferred chain. This code puts some outputs into message buffer, and then require a number from minibuffer.
Chain:
(deferred:$
(deferred:next
(lambda () (message "deferred start")))
(deferred:nextc it
(lambda ()
(message "chain 1")
1))
(deferred:nextc it
(lambda (x)
(message "chain 2 : %s" x)))
(deferred:nextc it
(lambda ()
(read-minibuffer "Input a number: ")))
(deferred:nextc it
(lambda (x)
(message "Got the number : %i" x)))
(deferred:error it
(lambda (err)
(message "Wrong input : %s" err))))
deferred:$
chains deferred objects.
it
holds a deferred object in the previous line.deferred:error
.After evaluating this s-exp and waiting for 1 second, a message is shown in the minibuffer.
Timer:
(deferred:$
(deferred:wait 1000) ; 1000msec
(deferred:nextc it
(lambda (x)
(message "Timer sample! : %s msec" x))))
This s-exp inserts the result that is performed by the command ls -la
. (This s-exp may not run in windows. Try dir
command.)
Command process:
(deferred:$
(deferred:process "ls" "-la")
(deferred:nextc it
(lambda (x) (insert x))))
This s-exp inserts a text from http://www.gnu.org asynchronously. (You can clear the result with undo command.)
HTTP GET:
(require 'url)
(deferred:$
(deferred:url-retrieve "http://www.gnu.org")
(deferred:nextc it
(lambda (buf)
(insert (with-current-buffer buf (buffer-string)))
(kill-buffer buf))))
This s-exp inserts an image from google asynchronously.
Get an image:
(deferred:$
(deferred:url-retrieve "http://www.google.co.jp/intl/en_com/images/srpr/logo1w.png")
(deferred:nextc it
(lambda (buf)
(insert-image
(create-image
(let ((data (with-current-buffer buf (buffer-string))))
(substring data (+ (string-match "\n\n" data) 2)))
'png t))
(kill-buffer buf))))
This s-exp retrieves two images from google concurrently and wait for the both results. Then, the file sizes of the images are inserted the current buffer.
Parallel deferred:
(deferred:$
(deferred:parallel
(lambda ()
(deferred:url-retrieve "http://www.google.co.jp/intl/en_com/images/srpr/logo1w.png"))
(lambda ()
(deferred:url-retrieve "http://www.google.co.jp/images/srpr/nav_logo14.png")))
(deferred:nextc it
(lambda (buffers)
(cl-loop for i in buffers
do
(insert
(format
"size: %s\n"
(with-current-buffer i (length (buffer-string)))))
(kill-buffer i)))))
deferred:parallel
runs asynchronous tasks concurrently.This s-exp executes following tasks:
Get an image by wget and resize by ImageMagick:
(deferred:$
;; try
(deferred:$
(deferred:process "wget" "-O" "a.jpg" "http://www.gnu.org/software/emacs/tour/images/splash.png")
(deferred:nextc it
(lambda () (deferred:process "convert" "a.jpg" "-resize" "100x100" "jpg:b.jpg")))
(deferred:nextc it
(lambda ()
(clear-image-cache)
(insert-image (create-image (expand-file-name "b.jpg") 'jpeg nil)))))
;; catch
(deferred:error it ;
(lambda (err)
(insert "Can not get a image! : " err)))
;; finally
(deferred:nextc it
(lambda ()
(deferred:parallel
(lambda () (delete-file "a.jpg"))
(lambda () (delete-file "b.jpg")))))
(deferred:nextc it
(lambda (x) (message ">> %s" x))))
Here is an another sample code for try-catch-finally blocks. This is simpler than above code because of the deferred:try' macro. (Note: They bring the same results practically, but are not perfectly identical. The
finally` task may not be called because of asynchrony.)
Try-catch-finally:
(deferred:$
(deferred:try
(deferred:$
(deferred:process "wget" "-O" "a.jpg" "http://www.gnu.org/software/emacs/tour/images/splash.png")
(deferred:nextc it
(lambda () (deferred:process "convert" "a.jpg" "-resize" "100x100" "jpg:b.jpg")))
(deferred:nextc it
(lambda ()
(clear-image-cache)
(insert-image (create-image (expand-file-name "b.jpg") `jpeg nil)))))
:catch
(lambda (err) (insert "Can not get a image! : " err))
:finally
(lambda ()
(delete-file "a.jpg")
(delete-file "b.jpg")))
(deferred:nextc it
(lambda (x) (message ">> %s" x))))
Although a long time command is executed (3 second sleeping), the task is rejected by timeout for 1 second.
The function deferred:earlier
also runs asynchronous tasks concurrently, however, the next deferred task receives the first result. The other results and tasks will be rejected (canceled or ignored).
Timeout Process:
(deferred:$
(deferred:earlier
(deferred:process "sh" "-c" "sleep 3 | echo 'hello!'")
(deferred:$
(deferred:wait 1000) ; timeout msec
(deferred:nextc it (lambda () "canceled!"))))
(deferred:nextc it
(lambda (x) (insert x))))
deferred:wait
, the next task receives a result of the command.deferred:parallel
and deferred:earlier
may be corresponding to and
and or
, respectively.Here is an another sample code for timeout, employing deferred:timeout
macro.
Timeout macro:
(deferred:$
(deferred:timeout
1000 "canceled!"
(deferred:process "sh" "-c" "sleep 3 | echo 'hello!'"))
(deferred:nextc it
(lambda (x) (insert x))))
Note that the deferred:timeout
and deferred:earlier
just rejects the task result and does not stop the running task chains. Please see the document for deferred:cancel
.
This s-exp plays an animation at the cursor position for few seconds. Then, you can move cursor freely, because the animation does not block Emacs.
Returning a deferred object in the deferred tasks, the returned task is executed before the next deferred one that is statically connected on the source code. (In this case, the interrupt task is dynamically connected.)
Employing a recursive structure of deferred tasks, you can construct a deferred loop. It may seem the multi-thread in Emacs Lisp.
Loop and animation:
(let ((count 0) (anm "-/|\\-")
(end 50) (pos (point))
(wait-time 50))
(deferred:$
(deferred:next
(lambda (x) (message "Animation started.")))
(deferred:nextc it
(deferred:lambda (x)
(save-excursion
(when (< 0 count)
(goto-char pos) (delete-char 1))
(insert (char-to-string
(aref anm (% count (length anm))))))
(if (> end (cl-incf count)) ; return nil to stop this loop
(deferred:nextc (deferred:wait wait-time) self)))) ; return the deferred
(deferred:nextc it
(lambda (x)
(save-excursion
(goto-char pos) (delete-char 1))
(message "Animation finished.")))))
deferred:lambda
is an anaphoric macro in which self
refers itself. It is convenient to construct a recursive structure.Let's say you have an asynchronous function which takes a callback. For example, dbus.el, xml-rpc.el and websocket.el has such kind of asynchronous APIs. To use such libraries with deferred.el, you can make an unregistered deferred object using deferred:new
and then start the deferred callback queue using deferred:callback-post
in the callback given to the asynchronous function. If the asynchronous function supports "errorback", you can use deferred:errorback-post
to pass the error information to the following callback queue.
In the following example, run-at-time
is used as an example for the asynchronous function. Deferred.el already has deferred:wait
for this purpose so that you don't need the following code if you want to use run-at-time
.
(deferred:$
(deferred:next
(lambda ()
(message "1")
1))
(deferred:nextc it
(lambda (x)
(let ((d (deferred:new #'identity)))
(run-at-time 0 nil (lambda (x)
;; Start the following callback queue now.
(deferred:callback-post d x))
x)
;; Return the unregistered (not yet started) callback
;; queue, so that the following queue will wait until it
;; is started.
d)))
;; You can connect deferred callback queues
(deferred:nextc it
(lambda (x)
(message "%s" (1+ x)))))
deferred:next (callback)
deferred:nextc (d callback)
deferred:error (d errorback)
deferred:cancel (d)
deferred:watch (d callback)
deferred:wait (msec)
deferred:$
it
holds a deferred object in the previous line.deferred:loop (number-or-list callback)
mapc
.deferred:parallel (list-or-alist)
deferred:earlier (list-or-alist)
deferred:cancel
)deferred:call (function args...)
funcall
deferred:apply (function args)
apply
deferred:process (command args...) / deferred:process-shell (command args...)
start-process
and start-process-shell-command
.deferred:process-buffer (command args...) / deferred:process-shell-buffer (command args...)
start-process
and start-process-shell-command
.deferred:wait-idle (msec)
deferred:url-retrieve (url [cbargs])
url-retrieve
in the url
package.[experimental] deferred:url-get (url [params])
[experimental] deferred:url-post (url [params])
deferred:new ([callback])
deferred:callback
or deferred:errorback
.deferred:wait
.)deferred:succeed ([value])
deferred:fail ([error])
deferred:callback (d [value])
deferred:callback-post (d [value])
deferred:errorback (d [error])
deferred:errorback-post (d [error])
deferred:try (d &key catch finally)
d
. (This function is expanded as an argument of deferred:error
.)d
finishes whether in success or failure. (This function is expanded as an argument of deferred:watch.)finally
task should be called.deferred:error
and deferred:watch
.deferred:timeout (msec timeout-form d)
d
.d
does not complete within timeout-msec
, this macro rejects the deferred task and return the timeout-form
. (See the document for deferred:cancel
)deferred:earlier
and deferred:wait
.deferred:process...
Some deferred functions can fire a deferred chain implicitly. Following functions register a deferred object with the execution queue to run asynchronously.
The deferred tasks those are created by deferred:new
are never called. Using this object, a deferred chain can pause to wait for other events. (See the source for deferred:wait
.)
One can fire the chain before connecting. That is, deferred objects wait for connecting the subsequent task holding the result value. The functions deferred:succeed
and deferred:fail
create those waiting objects.
The static connection (statically connected)
is a connection between deferred tasks on the source code.
This is a basic usage for the deferred chain.
The static connection is almost equivalent to ordinary callback notation as an argument in the function declarations. The deferred notation is easy to read and write better than the callback one, because the sequence of asynchronous tasks can be written by the deferred notation straightforward.
Returning a deferred object in the deferred tasks, the returned task is executed before the next deferred one that is statically connected on the source code. This is the dynamic connection (dynamically connected)
.
Employing a recursive structure of deferred tasks, you can construct higher level control structures, such as loop.
Some discussions of writing deferred codes.
Using the lexical scope macro, such as let
, the deferred tasks defined by lambdas can access local variables.
let
Ex.:
(let ((a (point)))
(deferred:$
(deferred:wait 1000)
(deferred:nextc it
(lambda (x)
(goto-char a)
(insert "here!")))))
If you write a code of deferred tasks without lexical scope macros, you should be careful with the scopes of each variables.
The excursion
functions those hold the current status with the s-exp form, such as save-execursion
or with-current-buffer
, are not valid in the deferred tasks, because of execution asynchronously.
Wrong Ex.:
(with-current-buffer (get-buffer "*Message*")
(deferred:$
(deferred:wait 1000)
(deferred:nextc it
(lambda (x)
(insert "Time: %s " x) ; `insert` may not be in the *Message* buffer!
))))
In this case, using lexical scope macros to access the buffer variable, you can change the buffer in the deferred task.
Corrected:
(let ((buf (get-buffer "*Message*")))
(deferred:$
(deferred:wait 1000)
(deferred:nextc it
(lambda (x)
(with-current-buffer buf ; Set buffer in the asynchronous task.
(insert "Time: %s " x))))))
However the dynamic connection is a powerful feature, sometimes it causes bugs of the wrong execution order, because of returning not intended deferred objects.
Then, you should watch the return values of the deferred tasks not to cause an unexpected dynamic connection.
The debugging of asynchronous tasks is difficult. Of course, you can use debugger for deferred tasks, but asynchronous tasks cause some troubles, such as interruptions of your debugging and timing gap of simultaneous deferred tasks. Therefore, logging is a safe debugging to observe the tasks correctly, for example, using the message
function and making custom application log buffer.
If deferred tasks fall into an infinite loop unexpectedly (but Emacs may not freeze), calling the command deferred:clear-queue
, you can stop the deferred tasks immediately.
If the errors occurred in deferred tasks are caught by no errorback functions, finally the deferred framework catches it and reports to the message buffer. Because the implementation of the framework uses a condition-case
form, the debugger can not catch the signals normally. If you want to debug the errors in the deferred tasks with the debug-on-error mechanism, set the variable deferred:debug-on-signal
non-nil.
Wrapping a deferred task in the function deferred:sync!
, you can wait for the result of the task synchronously. However, the wrapper function should be used for test or debug purpose, because the synchronous waiting is not exact.
Writing deferred tasks with deferred.el
, you may write a lot of deferred:nextc
and lambda
to define tasks. Defining a macro, you may write codes shortly. The test code test-deferred.el
uses many macros to shorten test codes.
On the other hand, using macros to hide lambda
, it is difficult to realize when the deferred codes are evaluated. That is why deferred.el
does not provide lot of convenient macros. If you use macros, be careful evaluation timing of deferred forms.
Following documents are good introduction to deferred.
(C) 2010-2016 SAKURAI Masashi All rights reserved. m.sakurai at kiwanami.net