imenu based fold parser

[wbolster]

hi,

i wrote a imenu based parser that creates folds (without any nesting) from imenu entries. i think this is very useful behaviour. let me know what you think, and whether this would make sense to include with origami itself.

i think it can be useful for many languages, e.g. #49, #29, #61, #58 and possibly others may benefit from this.

this parser can be used for any major mode that has sensible imenu entries, by putting it into the origami-parser-alist variable. this can be done in many ways, for instance via Customize M-x customize-variable RET origami-parser-alist, or using lisp code, which is what i do in my init.el using a helper from evil-mode:

(evil-add-to-alist 'origami-parser-alist 'python-mode 'origami-parser-imenu-flat)

i use this for python-mode and rst-mode. it results in very dense (no empty lines) and usable (for me at least) folding.

here's how it looks for python:

here's how it looks for restructuredtext:

here's the origami-parser-imenu-flat function, which works for me, but i only tested it casually:

  (defun origami-parser-imenu-flat (create)
    "Origami parser producing folds for each imenu entry, without nesting."
    (require 'imenu)
    (lambda (content)
      (let ((orig-major-mode major-mode))
        (with-temp-buffer
          (insert content)
          (funcall orig-major-mode)
          (let* ((items
                  (-as-> (imenu--make-index-alist t) items
                         (-flatten items)
                         (-filter 'listp items)))
                 (positions
                  (-as-> (-map #'cdr items) positions
                         (-filter 'identity positions)
                         (-map-when 'markerp 'marker-position positions)
                         (-filter 'natnump positions)
                         (cons (point-min) positions)
                         (-snoc positions (point-max))
                         (-sort '< positions)
                         (-uniq positions)))
                 (ranges
                  (-zip-pair positions (-map '1- (cdr positions))))
                 (fold-nodes
                  (--map
                   (-let*
                       (((range-beg . range-end) it)
                        (line-beg
                         (progn (goto-char range-beg)
                                (line-beginning-position)))
                        (offset
                         (- (min (line-end-position) range-end) line-beg))
                        (fold-node
                         (funcall create line-beg range-end offset nil)))
                     fold-node)
                   ranges)))
            fold-nodes)))))

[sawan]

Wow, this looks awesome.

So adding the elisp function + imenu is all that's required?

[wbolster]

yes, and adding the configuration so that it gets used for a specific major mode. and of course that major mode should have imenu support, which seems the case for quite a few that i tried.

[sawan]

I couldn't get this to work in Python mode. What entry function should I be invoking? I tried origami-close-all-nodes.

[wbolster]

@sawan what you describe works for me and is indeed how this is supposed to work.

what does your origami-parser-alist variable look like? does python-mode point to the right helper?

also try origami-reset, or killing the buffer and reopening the file.

[sawan]

@wbolster thanks, sorry for late reply.

I have added the function it to origami-parser-alist.

Now when I do origami-close-all-nodes I get an error saying let*: Symbol’s value as variable is void: create

[wbolster]

@sawan are you sure you set it up correctly?

i have a working config in my init.el which you may use for inspiration: https://github.com/wbolster/dotfiles/blob/master/Emacs/init.el:

(use-package origami
  :custom
  (origami-show-fold-header t)

  :commands
  origami-parser-imenu-flat

  :config
  (defun origami-parser-imenu-flat (create)
    "Origami parser producing folds for each imenu entry, without nesting."
    (lambda (content)
      (let ((orig-major-mode major-mode))
        (with-temp-buffer
          (insert content)
          (delay-mode-hooks
            (funcall orig-major-mode))
          (let* ((items
                  (-as-> (imenu--make-index-alist t) items
                         (-flatten items)
                         (-filter 'listp items)))
                 (positions
                  (-as-> (-map #'cdr items) positions
                         (-filter 'identity positions)
                         (-map-when 'markerp 'marker-position positions)
                         (-filter 'natnump positions)
                         (cons (point-min) positions)
                         (-snoc positions (point-max))
                         (-sort '< positions)
                         (-uniq positions)))
                 (ranges
                  (-zip-pair positions (-map '1- (cdr positions))))
                 (fold-nodes
                  (--map
                   (-let*
                       (((range-beg . range-end) it)
                        (line-beg
                         (progn (goto-char range-beg)
                                (line-beginning-position)))
                        (offset
                         (- (min (line-end-position) range-end) line-beg))
                        (fold-node
                         (funcall create line-beg range-end offset nil)))
                     fold-node)
                   ranges)))
            fold-nodes))))))

[sawan]

Hi @wbolster,

I have added the parser function to my configuration:

https://github.com/sawan/emacs-config/blob/279e125344a81d3bf548e64c302829bc3a2efcb2/emacs24.el#L1614-L1651

And here is the value of origami-parser-alist

https://github.com/sawan/emacs-config/blob/279e125344a81d3bf548e64c302829bc3a2efcb2/emacs24.el#L1718-L1746

Am I doing anything wrong?

[gregsexton]

Great idea! Do you want to send a pull request?

[wbolster]

@sawan that is also how i use it. not sure what's wrong at your side, do you have the latest origami? and can you provide more info like a backtrace (M-x toggle-debug-on-error)?

[wbolster]

@gregsexton sure, would that pr just include that function? or also change default configuration? (imenu and indent based folds are different.)

[sawan]

@wbolster I have sent you an email with the stack trace attached.

Origami version is 20180101.753

Thanks again....

[wbolster]

@sawan i am not sure which email you are referring to... in any case, the only right place to put that stack trace is as a comment in this ticket.

[wbolster]

ah, found something. here's a trimmed down version

Debugger entered--Lisp error: (void-variable create)
(funcall create line-beg range-end offset nil)
(let* ((--dash-source-31-- it) (range-beg (car-safe (prog1 --dash-source-31-- (setq --dash-source-31-- (cdr --dash-source-31--))))) (range-end --dash-source-31--) (line-beg (progn (goto-char range-beg) (line-beginning-position))) (offset (- (min (line-end-position) range-end) line-beg)) (fold-node (funcall create line-beg range-end offset nil))) fold-node)
(lambda (it) (let* ((--dash-source-31-- it) (range-beg (car-safe (prog1 --dash-source-31-- (setq --dash-source-31-- (cdr --dash-source-31--))))) (range-end --dash-source-31--) (line-beg (progn (goto-char range-beg) (line-beginning-position))) (offset (- (min (line-end-position) range-end) line-beg)) (fold-node (funcall create line-beg range-end offset nil))) fold-node))((1 . 1746))
mapcar((lambda (it) (let* ((--dash-source-31-- it) (range-beg (car-safe (prog1 --dash-source-31-- (setq --dash-source-31-- (cdr --dash-source-31--))))) (range-end --dash-source-31--) (line-beg (progn (goto-char range-beg) (line-beginning-position))) (offset (- (min (line-end-position) range-end) line-beg)) (fold-node (funcall create line-beg range-end offset nil))) fold-node)) ((1 . 1746) (1747 . 2334) (2335 . 3233) (3234 . 3287) (3288 . 3499) (3500 . 4360)))
(let* ((items (let ((items (imenu--make-index-alist t))) (let ((items (-flatten items))) (-filter (quote listp) items)))) (positions (let ((positions (-map (function cdr) items))) (let ((positions (-filter ... positions))) (let ((positions ...)) (let (...) (let ... ...)))))) (ranges (-zip-pair positions (-map (quote 1-) (cdr positions)))) (fold-nodes (mapcar (function (lambda (it) (let* (... ... ... ... ... ...) fold-node))) ranges))) fold-nodes)
(progn (insert content) (funcall orig-major-mode) (let* ((items (let ((items (imenu--make-index-alist t))) (let ((items ...)) (-filter (quote listp) items)))) (positions (let ((positions (-map ... items))) (let ((positions ...)) (let (...) (let ... ...))))) (ranges (-zip-pair positions (-map (quote 1-) (cdr positions)))) (fold-nodes (mapcar (function (lambda (it) (let* ... fold-node))) ranges))) fold-nodes))
(unwind-protect (progn (insert content) (funcall orig-major-mode) (let* ((items (let ((items ...)) (let (...) (-filter ... items)))) (positions (let ((positions ...)) (let (...) (let ... ...)))) (ranges (-zip-pair positions (-map (quote 1-) (cdr positions)))) (fold-nodes (mapcar (function (lambda ... ...)) ranges))) fold-nodes)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert content) (funcall orig-major-mode) (let* ((items (let (...) (let ... ...))) (positions (let (...) (let ... ...))) (ranges (-zip-pair positions (-map ... ...))) (fold-nodes (mapcar (function ...) ranges))) fold-nodes)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))
(let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert content) (funcall orig-major-mode) (let* ((items (let ... ...)) (positions (let ... ...)) (ranges (-zip-pair positions ...)) (fold-nodes (mapcar ... ranges))) fold-nodes)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))))
(let ((orig-major-mode major-mode)) (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert content) (funcall orig-major-mode) (let* ((items ...) (positions ...) (ranges ...) (fold-nodes ...)) fold-nodes)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))))
(lambda (content) (let ((orig-major-mode major-mode)) (let
((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer
(set-buffer temp-buffer) (unwind-protect (progn (insert content)
(funcall orig-major-mode) (let* (... ... ... ...) fold-nodes)) (and
(buffer-name temp-buffer) (kill-buffer temp-buffer))))))) ...
funcall((lambda (content) (let ((orig-major-mode major-mode)) (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert content) (funcall orig-major-mode) (let* (... ... ... ...) fold-nodes)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))))) ...)
(origami-fold-root-node (funcall parser contents))
(let ((contents (buffer-string))) (origami-fold-root-node (funcall parser contents)))
(save-current-buffer (set-buffer buffer) (let ((contents (buffer-string))) (origami-fold-root-node (funcall parser contents))))
(progn (save-current-buffer (set-buffer buffer) (let ((contents (buffer-string))) (origami-fold-root-node (funcall parser contents)))))
(if parser (progn (save-current-buffer (set-buffer buffer) (let ((contents (buffer-string))) (origami-fold-root-node (funcall parser contents))))))
origami-build-tree(#<buffer aota.py> (lambda (content) (let ((orig-major-mode major-mode)) (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert content) (funcall orig-major-mode) (let* (... ... ... ...) fold-nodes)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))))))
(if (origami-rebuild-tree\? buffer) (origami-build-tree buffer (origami-get-parser buffer)) (origami-get-cached-tree buffer))
(progn (if (origami-rebuild-tree\? buffer) (origami-build-tree buffer (origami-get-parser buffer)) (origami-get-cached-tree buffer)))
(if origami-mode (progn (if (origami-rebuild-tree\? buffer) (origami-build-tree buffer (origami-get-parser buffer)) (origami-get-cached-tree buffer))))
origami-get-fold-tree(#<buffer aota.py>)
(let ((tree (origami-get-fold-tree buffer))) (if tree (progn (origami-apply-new-tree buffer tree (origami-store-cached-tree buffer (origami-fold-map (function (lambda ... ...)) tree))))))
origami-close-all-nodes(#<buffer aota.py>)
funcall-interactively(origami-close-all-nodes #<buffer aota.py>)
#<subr call-interactively>(origami-close-all-nodes record nil)
apply(#<subr call-interactively> origami-close-all-nodes (record nil))
(let ((ido-cr+-current-command command)) (apply orig-fun command args))
call-interactively@ido-cr+-record-current-command(#<subr call-interactively> origami-close-all-nodes record nil)
apply(call-interactively@ido-cr+-record-current-command #<subr call-interactively> (origami-close-all-nodes record nil))
call-interactively(origami-close-all-nodes record nil)
command-execute(origami-close-all-nodes record)
execute-extended-command(nil "origami-close-all-nodes")

[wbolster]

weird, that create variable is an argument to the parser function, and should be passed in by origami.

which emacs version?

and no idea whether it's related, but do you have lexical-binding on? e.g. first line of my init.el reads

;;; init.el --- emacs configuration -*- lexical-binding: t; -*-

[sawan]

@wbolster

Lexical Binding was indeed the problem!!

I put

;;; -*- lexical-binding: t -*-

in my .emacs and it works like a charm -- may I say its a very good approach to solving this problem!

Thank you again and I hope you keep up the good work.

[wbolster]

@sawan awesome!

[jimmywongroo]

@wbolster Care to open a PR for this useful feature?

[wbolster]

tbh i'm not sure this project is actually maintained. @gregsexton is it?

[jcs090218]

@gregsexton Are you still maintaining this project, if you are just busy consider adding some collaborator or something that could keep this project maintained! Please keep this awesome project alive! Thank you!

[gregsexton]

I like origami-parser-imenu-flat, I've been using it for a few months without a problem. Will merge a PR.

[jcs090218]

Hi, just want inform people here that have started a new branch here.

If you don't mind using celpa, you can open the PR there instead! Thanks!

For reason why? See https://github.com/jcs-elpa/origami.el/issues/1.