Optimization for directory-files

[alphapapa]

Hi there,

In this interesting discussion with Alexander Miller, he mentioned that directory-files is faster when called in a temp buffer. This quick test seems to show that it is indeed:

#+BEGIN_SRC elisp
  (defun f--collect-entries--tb (path recursive)
    (with-temp-buffer
      (let (result
            (entries
             (-reject
              (lambda (file)
                (or
                 (equal (f-filename file) ".")
                 (equal (f-filename file) "..")))
              (directory-files path t))))
        (cond (recursive
               (-map
                (lambda (entry)
                  (if (f-file? entry)
                      (setq result (cons entry result))
                    (when (f-directory? entry)
                      (setq result (cons entry result))
                      (setq result (append result (f--collect-entries entry recursive))))))
                entries))
              (t (setq result entries)))
        result)))

  (list (cons "without temp buffer"
              (progn
                (garbage-collect)
                (benchmark-run-compiled 100
                  (f--collect-entries "~/.emacs.d" nil))))
        (cons "with temp buffer"
              (progn
                (garbage-collect)
                (benchmark-run-compiled 100
                  (f--collect-entries--tb "~/.emacs.d" nil)))))
#+END_SRC

#+RESULTS:
| without temp buffer | 0.31999160200000004 | 0 | 0.0 |
| with temp buffer    |         0.191897104 | 0 | 0.0 |

I thought I should let you know in so that it could be added to f--collect-entries. Seems like a nice, free optimization. :)

Thanks for your work on f.el!

[alphapapa]

@rejeep I rediscovered this in my notes, came here to repost it, and found that I already had. :) Here's another benchmark using bench-multi:

(defun f--collect-entries--tb (path recursive)
  (with-temp-buffer
    (let (result
          (entries
           (-reject
            (lambda (file)
              (or
               (equal (f-filename file) ".")
               (equal (f-filename file) "..")))
            (directory-files path t))))
      (cond (recursive
             (-map
              (lambda (entry)
                (if (f-file? entry)
                    (setq result (cons entry result))
                  (when (f-directory? entry)
                    (setq result (cons entry result))
                    (setq result (append result (f--collect-entries entry recursive))))))
              entries))
            (t (setq result entries)))
      result)))

#+BEGIN_SRC el
  (bench-multi :times 100
    :ensure-equal t
    :forms (("without temp buffer" (f--collect-entries "~/.emacs.d" nil))
            ("with temp buffer" (f--collect-entries--tb "~/.emacs.d" nil))))
#+END_SRC

Form	x faster than next	Total runtime	# of GCs	Total GC runtime
with temp buffer	1.66	0.087498083	0	0.0
without temp buffer	slowest	0.14550272800000003	0	0.0

[Phundrak]

Hi, I can’t reproduce these results, in fact using a temporary buffer makes the function slower. This is what I get when I execute the code you shared in your first comment:

without temp buffer	0.031098385	0	0.0
with temp buffer	0.033553346	0	0.0

Can you still reproduce your results?

[alphapapa]

Can you still reproduce your results?

I don't know, that was nearly four years ago, and Emacs has had a few releases since then. What version did you test on? Were you using emacs -q?

[Phundrak]

I am currently on Emacs 28.1 and on master. The results I shared above are from master, below are the results I get on Emacs 28.1 on an Intel Celeron N2840 (2.16GHz max):

without temp buffer 0.104090848 0 0.0 with temp buffer 0.11387473000000001 0 0.0

I guess this hack used to be useful but that is no longer the case.

-- Lucien “Phundrak” Cartier-Tilet

https://phundrak.com (Français)

https://phundrak.com/en (English)

Sent from GNU/Emacs