ob-ein :post header argument gets overridden after async results is returned

[yatsky]

Hi,

I'm trying to use :post argument in my ein-python block to process the results from the block, but it is only applied to *ob-ein-sentinel*, instead of the async result that replaces *ob-ein-sentinel*. I've managed to find roughly where to change in ob-ein.el and got the code from ob-core.el that's responsible for post-processing the result, but I don't know elisp well enough to make it work.

Here's what I want to achieve

I've found the relevant code in ob-ein.el

(defun ob-ein--execute-async-callback (buffer params result-type result-params name)
  "Return callback of 1-arity (the shared output cell) to update org buffer when
`ein:shared-output-eval-string' completes.

The callback returns t if results containt RESULT-TYPE outputs, nil otherwise."
  (apply-partially
   (lambda (buffer* params* result-type* result-params* name* cell)
     (when-let ((raw (aif (ein:oref-safe cell 'traceback)
             (ansi-color-apply (ein:join-str "\n" it))
               (ob-ein--process-outputs result-type* cell params*))))
       (prog1 t
     (let ((result
        (let ((tmp-file (org-babel-temp-file "ein-")))
          (with-temp-file tmp-file (insert raw))
          (org-babel-result-cond result-params*
            raw (org-babel-import-elisp-from-file tmp-file '(16)))))
           (info (org-babel-get-src-block-info 'light)))
       (ein:log 'debug "ob-ein--execute-async-callback %s \"%s\" %s"
            name* result buffer*)
       (save-excursion
         (save-restriction
           (with-current-buffer buffer*
         (unless (stringp (org-babel-goto-named-src-block name*)) ;; stringp=error
           (when (version-list-< (version-to-list (org-release)) '(9))
             (when info ;; kill #+RESULTS: (no-name)
               (setf (nth 4 info) nil)
               (org-babel-remove-result info))
             (org-babel-remove-result)) ;; kill #+RESULTS: name
           (org-babel-insert-result
                   ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                   ;; I think I just need to update this `result` using the code from `org-babel-execute-src-block` in `ob-core.el`
                    ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
            result
            (cdr (assoc :result-params
                (cl-third (org-babel-get-src-block-info)))))
           (org-redisplay-inline-images)))))))))
   buffer params result-type result-params name))

Code from org-babel-execute-src-block in ob-core.el for post-processing the result.

        ;; Possibly perform post process provided its
        ;; appropriate.  Dynamically bind "*this*" to the
        ;; actual results of the block.
        (let ((post (cdr (assq :post params))))
          (when post
            (let ((*this* (if (not file) result
                    (org-babel-result-to-file
                     file
                     (let ((desc (assq :file-desc params)))
                       (and desc (or (cdr desc) result)))))))
              (setq result (org-babel-ref-resolve post))
              (when file
            (setq result-params (remove "file" result-params))))))

Thank you!

[dickmao]

I thought about doing this via add-function and it was so convoluted that it's cleaner just to rewrite ob-ein--execute-async-callback in precisely the way you described (but you would need to call it :async-post so as not to have it apply to *ob-ein-sentinel*. If you are particularly inspired you might even consider making a PR out of the thing with tests and all (doubtful).

[yatsky]

I thought about doing this via add-function and it was so convoluted that it's cleaner just to rewrite ob-ein--execute-async-callback in precisely the way you described (but you would need to call it :async-post so as not to have it apply to *ob-ein-sentinel*. If you are particularly inspired you might even consider making a PR out of the thing with tests and all (doubtful).

Thanks for the quick reply and confirming this @dickmao. I'll try at least getting something working for myself with my limited elisp knowledge.

[yatsky]

A hack that works for now. Just in case anyone else's looking for the same thing.

(defun ob-ein--execute-async-callback (buffer params result-type result-params name)
  "Return callback of 1-arity (the shared output cell) to update org buffer when
`ein:shared-output-eval-string' completes.

The callback returns t if results containt RESULT-TYPE outputs, nil otherwise."
  (apply-partially
   (lambda (buffer* params* result-type* result-params* name* cell)
     (when-let ((raw (aif (ein:oref-safe cell 'traceback)
             (ansi-color-apply (ein:join-str "\n" it))
               (ob-ein--process-outputs result-type* cell params*))))
       (prog1 t
     (let ((result
        (let ((tmp-file (org-babel-temp-file "ein-")))
          (with-temp-file tmp-file (insert raw))
          (org-babel-result-cond result-params*
            raw (org-babel-import-elisp-from-file tmp-file '(16)))))
           (info (org-babel-get-src-block-info 'light)))
       (ein:log 'debug "ob-ein--execute-async-callback %s \"%s\" %s"
            name* result buffer*)
       (save-excursion
         (save-restriction
           (with-current-buffer buffer*
         (unless (stringp (org-babel-goto-named-src-block name*)) ;; stringp=error
           (when (version-list-< (version-to-list (org-release)) '(9))
             (when info ;; kill #+RESULTS: (no-name)
               (setf (nth 4 info) nil)
               (org-babel-remove-result info))
             (org-babel-remove-result)) ;; kill #+RESULTS: name

                   ;; Possibly perform async-post process provided its
                   ;; appropriate.  Dynamically bind "*this*" to the
                   ;; actual results of the block.
                   ;; Also need to handle the scenario when the code block from :async-post
                   ;; cannot be found.
                   (let ((async-post (cdr (assq :async-post params))))
                     (when async-post
                       (setq *this* result)
                       (setq result (org-babel-ref-resolve async-post))))
           (org-babel-insert-result
            result
            (cdr (assoc :result-params
                (cl-third (org-babel-get-src-block-info)))))
           (org-redisplay-inline-images)))))))))
   buffer params result-type result-params name))