Replace evil -- first pass

[devcarbon-com]

Summary of Changes

I got a little carried away while replacing evil-surround and replaced evil altogether :P

Draft.

Initial testing looks promising.

Still need to update the doc strings everywhere that refer to evil specifically where they could be modal in general.

When merged will resolve #106.

Public Domain Dedication

• [X] In contributing, I relinquish any copyright claims on my contribution and freely release it into the public domain in the simple hope that it will provide value.

(Why: The freely released, copyright-free work in this repository represents an investment in a better way of doing things called attribution-based economics. Attribution-based economics is based on the simple idea that we gain more by giving more, not by holding on to things that, truly, we could only create because we, in our turn, received from others. As it turns out, an economic system based on attribution -- where those who give more are more empowered -- is significantly more efficient than capitalism while also being stable and fair (unlike capitalism, on both counts), giving it transformative power to elevate the human condition and address the problems that face us today along with a host of others that have been intractable since the beginning. You can help make this a reality by releasing your work in the same way -- freely into the public domain in the simple hope of providing value. Learn more about attribution-based economics at drym.org, tell your friends, do your part.)

[countvajhula]

Nice @devcarbon-com ! I don't have time to look at this in detail right now, but my first thoughts are:

• Rigpa assumes that Symex is an evil state. Many users use Symex with Rigpa (including myself), so in order to eliminate evil we need to have a solution here. One option could be to just trigger the evil state transition even if we don't actually use any other features of evil (an empty evil "shell"), for the initial pass (i.e. for 2.0), and then move that layer into a symex-evil package later. A second option could be to have a fallback modal interface handler in Rigpa that uses whatever abstraction of a modal interface you may have put in place here.
• There are also other integrations with telephone-line and powerline that provide some UI feedback that the user is in Symex mode. This is not a high priority, but it's another thing to keep in mind that Evil gives us "for free" -- once again, this could be something we move into a symex-evil package down the line but if we don't have a solution on day 1 for 2.0, I'd favor having a shell Evil layer in place just to keep things working as they were before until we can transition users properly to symex-evil.

[devcarbon-com]

Good to know.

I've not replaced evil completely, but I have made it's loading optional. I haven't tested this specifically yet, but I think it should still work with Rigpa and Evil users.

Personally I'm aiming to make this work with Meow, but there are several modal options out there and I think it would be neat if it were trivial to use Symex with any of them.

I think the main missing piece at the moment is enter lower/lowest and higher. Perhaps the last case statement for each of those could call out to a user customizable function.

[devcarbon-com]

Seems to be working pretty well so far.

Here is my current (WIP) setup with meow in case anyone wants to try it out: (note that I started with the meow defaults for the dvorak layout)

(use-package symex
  :load-path "~/projects/symex.el"
  :config

  (setq symex-highlight-p nil)

  (defun symex-user-defined-higher-mode ()
    (interactive)
    (if dc/return-to-symex
        (prog2 (setq-local dc/return-to-symex nil)
            (meow-enter-paren-mode))
      (meow-enter-normal-mode)))

  (defun symex-user-defined-lower-mode ()
    (setq-local dc/return-to-symex t)
    (meow-insert-mode))

  (defun symex-user-defined-lowest-mode () (symex-user-defined-lower-mode)))

;;* Modal user interface

(use-package meow
  :demand t
  :config
  (defvar dc/return-to-symex nil "local-var used to know if we've entered insert from paren mode, and if so, return there.")
  (defun meow-enter-normal-mode () (interactive) (meow-normal-mode 1))
  (defun meow-enter-paren-mode ()
    (interactive)
    (meow-paren-mode 1)
    (symex-select-nearest-in-line)
    (symex--adjust-point)
    (symex-initialize))
  (setq meow-paren-keymap (make-keymap))
  (meow-define-state paren
    "meow state for interacting with smartparens"
    :lighter " [P]"
    :keymap meow-paren-keymap)

    ;; meow-define-state creates the variable
  (setq meow-cursor-type-normal 'hollow)
  (setq meow-cursor-type-paren 'box)
  (meow-define-keys 'paren
    '("0" . digit-argument)
    '("9" . digit-argument)
    '("8" . digit-argument)
    '("7" . digit-argument)
    '("6" . digit-argument)
    '("5" . digit-argument)
    '("4" . digit-argument)
    '("3" . digit-argument)
    '("2" . digit-argument)
    '("1" . digit-argument)

    '("<backspace>" . dc/switch-to-previous-buffer)
    '("q" . meow-quit)
    '("Q" . kill-buffer-and-window)
    '("h" . symex-go-backward)
    '(")" . symex-wrap-round)
    '("]" . symex-wrap-square)
    '("C-'" . symex-cycle-quote)
    '("C-," . symex-cycle-unquote)
    '("`" . symex-add-quoting-level)
    '("C-`" . symex-remove-quoting-level)
    '("y" . symex-yank)
    '("K" . symex-yank-remaining)
    '("p" . symex-paste-after)
    '("P" . symex-paste-before)
    '("x" . symex-delete)
    '("X" . symex-delete-backwards)
    '("D" . symex-delete-remaining)
    '("c" . symex-change)
    '("C" . symex-change-remaining)
    '("C--" . symex-clear)
    '("M-h" . symex-change-delimiter)
    '("H" . symex-shift-backward)
    '("S" . symex-shift-forward)
    '("M-H" . symex-shift-backward-most)
    '("M-S" . symex-shift-forward-most)
                                        ; revisit kb
    '("M-(" . symex-emit-backward)
    '("(" . symex-capture-backward)
    '(")" . symex-capture-forward)
    '("M-)" . symex-emit-forward)
    '("z" . symex-swallow)
    '("Z" . symex-swallow-tail)
    '("e" . symex-evaluate)
    '("E" . symex-evaluate-remaining)
    '("C-M-e" . symex-evaluate-pretty)
    '("d" . symex-evaluate-definition)
    '("M-e" . symex-eval-recursive)
    '("|" . symex-split)
    '("_" . symex-split)
    '("&" . symex-join)
    '("-" . symex-splice)
    '("o" . symex-open-line-after)
    '("O" . symex-open-line-before)
    '(">" . symex-insert-newline)
    '("<" . symex-join-lines-backwards)
    '("C->" . symex-append-newline)
    '("C-<" . symex-join-lines)
    '("C-S-o" . symex-append-newline)
    '("J" . symex-join-lines)
    '("M-J" . symex-collapse)
    '("M-<" . symex-collapse)
    '("M->" . symex-unfurl)
    '("C-M-<" . symex-collapse-remaining)
    '("C-M->" . symex-unfurl-remaining)
    '("0" . symex-goto-first)
    '("M-h" . symex-goto-first)
    '("$" . symex-goto-last)
    '("M-l" . symex-goto-last)
    '("M-j" . symex-goto-lowest)
    '("M-k" . symex-goto-highest)
    '("=" . symex-tidy)
    '("<tab>" . symex-tidy)
    '("C-=" . symex-tidy-remaining)
    '("C-<tab>" . symex-tidy-remaining)
    '("M-=" . symex-tidy-proper)
    '("M-<tab>" . symex-tidy-proper)
    '("A" . symex-append-after)
    '("a" . symex-insert-at-end)
    '("i" . meow-insert-mode)
    '("I" . symex-insert-before)
    '("w" . symex-wrap)
    '("W" . symex-wrap-and-append)
    '(";" . symex-comment)
    '("M-;" . symex-comment-remaining)
    '("C-;" . symex-eval-print) ; weird pre-offset (in both)
    '("H-h" . symex--toggle-highlight) ; treats visual as distinct mode
    '("C-?" . symex-describe)
    '("<return>" . symex-enter-lower)

    '("d" . symex-go-up) ; Symex is in terms of branches as going up as a tree, whereas I think of branches of a root system.
    '("u" . symex-go-down) ; Therefore I want to visually go down a line, not up a limb to climb a "branch".
    '("s" . symex-go-forward)
    '("n" . symex-traverse-forward)
    '("N" . symex-traverse-forward-skip)
    '("C-w" . symex-wrap-square)
    '("M-w" . symex-wrap-curly)
    '("M-d" . symex-climb-branch)
    '("M-u" . symex-descend-branch)
    '("M-j" . symex-goto-highest)
    '("M-k" . symex-goto-lowest)
    '("." . symex-leap-forward)
    '("," . symex-leap-backward)
    '("r" . paredit-raise-sexp)

    '("<escape>" . meow-enter-normal-mode))

  (setq meow-cheatsheet-layout meow-cheatsheet-layout-dvorak)

  (meow-leader-define-key
   '("1" . meow-digit-argument)
   '("2" . meow-digit-argument)
   '("3" . meow-digit-argument)
   '("4" . meow-digit-argument)
   '("5" . meow-digit-argument)
   '("6" . meow-digit-argument)
   '("7" . meow-digit-argument)
   '("8" . meow-digit-argument)
   '("9" . meow-digit-argument)
   '("0" . meow-digit-argument)
   '("/" . meow-keypad-describe-key)
   '("?" . meow-cheatsheet))

  (meow-motion-overwrite-define-key
   ;; custom keybinding for motion state
   '("<escape>" . meow-enter-normal-mode))

  (meow-normal-define-key
   '("<backspace>" . dc/switch-to-previous-buffer)
   '("-" . negative-argument)
   '(";" . meow-reverse)
   '("," . meow-inner-of-thing)
   '("." . meow-bounds-of-thing)
   '("<" . meow-beginning-of-thing)
   '(">" . meow-end-of-thing)
   '("a" . meow-append)
   '("A" . meow-open-below)
   '("b" . meow-back-word)
   '("B" . meow-back-symbol)
   '("c" . meow-change)
   '("d" . meow-delete)
   '("D" . meow-backward-delete)
   '("e" . meow-line)
   '("E" . meow-goto-line)
   '("f" . meow-find)
   '("g" . meow-cancel-selection)
   '("G" . meow-grab)
   '("h" . meow-left)
   '("H" . meow-left-expand)
   '("i" . meow-insert)
   '("I" . meow-open-above)
   '("j" . meow-join)
   '("k" . meow-kill)
   '("K" . meow-enter-paren-mode)
   '("l" . meow-till)
   '("m" . meow-mark-word)
   '("M" . meow-mark-symbol)
   '("n" . meow-next)
   '("N" . meow-next-expand)
   '("o" . meow-block)
   '("O" . meow-to-block)
   '("p" . meow-prev)
   '("P" . meow-prev-expand)
   '("q" . meow-quit)
   '("Q" . kill-buffer-and-window)
   '("r" . meow-replace)
   '("R" . meow-swap-grab)
   '("s" . meow-search)
   '("t" . meow-right)
   '("T" . meow-right-expand)
   '("u" . meow-undo)
   '("U" . meow-undo-in-selection)
   '("v" . meow-visit)
   '("w" . meow-next-word)
   '("W" . meow-next-symbol)
   '("x" . meow-save)
   '("X" . meow-sync-grab)
   '("y" . meow-yank)
   '("z" . meow-pop-selection)
   '("'" . repeat)
   '("<escape>" . meow-enter-paren-mode))

  (meow-define-keys 'insert
    '("<escape>" . symex-user-defined-higher-mode))

  (meow-global-mode 1))

[countvajhula]

This is a good start! A few initial comments:

This PR disables evil wholesale, but Evil plays two distinct roles in the code that call for distinct handling. The first role is in the core, as a shortcut to implement features that we would otherwise need custom implementations for. The second role is as a modal interface, providing users a way to access Symex features whose implementation is a black box.

For the core, we'd like to eliminate evil entirely, by using built-in Emacs or custom implementations instead. For the most part, this is what you've done here, but there are places, like the "emacslike" and "normallike" states, where Evil functionality is simply conditionally disabled. I don't recall exactly what the purpose of these was, but without delving into that for the moment, I'm assuming that disabling it is likely to break something when the affected features are used in Meow. Looks like that's used in symex-evaluate -- anything broken there? In any event, it would be ideal to eliminate Evil if possible since that's what we'd need to do eventually anyway.

For the modal interface, disabling Evil functionality means that users who don't have Evil installed would be left without a modal interface. This is in contrast to today where even vanilla Emacs users get a modal interface out of the box (that happens to be implemented in Evil, but this is abstracted from them).

I'd recommend creating a new symex-meow module (see symex-evil and symex-hydra (on the master branch) for examples). That way, users can indicate they want to use meow via a defcustom (again, the master branch, which abstracts over the modal interface and has two distinct implementations in evil and hydra, can provide examples -- see this defcustom, although, the master branch doesn't actually avoid loading the hydra and evil dependencies, so guarding those imports with no-error, as you've done, looks good). I think your meow user configuration above should prove a good start for this symex-meow module.

With this in place, we could remove symex-evil, symex-meow (and maybe reintroduce symex-hydra), as part of the broader effort to decouple things in #26 (which is likely post-2.0 to keep this release's scope manageable). Alternatively, you could implement symex-meow as a separate package even today, if you think that would be easier. Lastly, for the purposes of this PR, I think we'd want to retain evil as a dependency in Symex for the moment, and retain the default symex-modal-backend value at evil, to preserve the existing behavior of providing a modal interface out of the box for all users.

[devcarbon-com]

@countvajhula

This PR disables evil wholesale, ....

Not quite what I had in mind; just to make it a soft dependency instead of a hard dependency. If evil is available it is intended to be used. If not, I've messed something up in the PR.

Let me check ........ Yep: Just checked with emacs -Q, looks like I have indeed messed something up! I'll see if I can track down why.

there are places, like the "emacslike" and "normallike" states, where Evil functionality is simply conditionally disabled. ... ... I'm assuming that disabling it is likely to break something when the affected features are used in Meow. Looks like that's used in symex-evaluate -- anything broken there?

If I understand correctly, emacslike and normallike were added to work around problems with hooks that evil adds.

This allows changes "under evil's radar" so that there are not unwanted side-effects. No evil, no added hooks to avoid, no problem.

(That's the theory, at least :P. Anecdotally I haven't had any trouble with symex-evaluate and I've been running this PR with meow for about a week.)

For the modal interface, disabling Evil functionality means that users who don't have Evil installed would be left without a modal interface. This is in contrast to today where even vanilla Emacs users get a modal interface out of the box (that happens to be implemented in Evil, but this is abstracted from them).

Do you mean by this that long-term, symex is intended to have it's own modal interface independent of third-party modal interfaces?

I'd recommend creating a new symex-meow module ... That way, users can indicate they want to use meow via a defcustom

Oh, I like this idea!

... I think we'd want to retain evil as a dependency in Symex for the moment, and retain the default symex-modal-backend value at evil, to preserve the existing behavior of providing a modal interface out of the box for all users.

Sounds good, I'll see if I can't fix this PR so that it works with Evil again.

[countvajhula]

Ah yes, sorry, I didn't mean to suggest that you had removed evil for evil users, just that you had disabled it for non-evil users, and that that may lose functionality in the core. But it sounds like that may not be the case.

Btw, I should have asked earlier -- is all you're trying to do use Symex with Meow? If so, then just implementing symex-meow, mirroring the symex-hydra that was recently removed, should accomplish that 😄 Of course, I appreciate your broader effort at decoupling evil here.

[countvajhula]

Do you mean by this that long-term, symex is intended to have it's own modal interface independent of third-party modal interfaces?

Long term, it would look more like #26 , so no, a modal interface would not be bundled. It would be up to the user to install either symex-evil, or symex-meow/hydra/anything else if they want a modal UI. Meanwhile, library authors could just use the DSL without any of the UI stuff, or even just the tree-sitter component without any of the Lisp stuff, and could mix and match as needed (also related: #9 ).

[devcarbon-com]

Nope, not all I'm after. Main thing is that I like having light dependencies, and evil just doesn't qualify.

Really I'm not using Meow for much of anything, other than to replace evil as the modal backend for Symex :)

[devcarbon-com]

Aha, found problem. Accidentally put (when (symex--evil-enabled-p) instead of (when (symex--evil-installed-p) .

Now this works in emacs -Q

(require 'package)
(add-to-list 'package-archives
             '("melpa" . "https://melpa.org/packages/") t)
(package-initialize)

(use-package evil :ensure t)
(use-package tsc :ensure t)
(use-package tree-sitter :ensure t)

(use-package symex
  :load-path "~/projects/symex.el") ;; path to local mirror of this PR

(symex-initialize)

[countvajhula]

@devcarbon-com To set expectations on the timelines on this work, I think eliminating the Evil dependency is premature at this stage and is going to require more structured handling. For now, I think it is safe to merge:

1. The changes that eliminate evil-surround
2. The changes which eliminate reliance on Evil interfaces by providing alternative/native implementations

As far as the conditional checks for evil-installed-p and stubbed interfaces, although it solves the dependency issue in your specific case, it introduces unnecessary complexity at this stage that doesn't help us truly address the problem. At a high level, I'd love to understand why Evil is used in the core (i.e. non-UI parts) at all, and either eliminate it, or fully encapsulate it in some kind of "adapter" module / layer. But as eliminating Evil isn't in scope for the 2.0 release, I don't anticipate being able to consider these changes in the immediate future. If you're fine with that, we could continue working on this PR on a longer timeframe, as long as you realize it will probably be after 2.0!

But a better option (which I would recommend) is if you could trim the PR to the enumerated safe changes specifically, we could merge them in now before tackling the other stuff so that they aren't held up by longer term discussions. We could then continue the discussion on Evil on a dedicated second PR.

Wdyt?

[devcarbon-com]

@countvajhula Sounds good!

I've got deadline I'm racing against right now with a project at work. but I'll get to this as soon as I'm able.

[countvajhula]

Hey @devcarbon-com ! Just a heads up I may be doing some refactoring in the near future that may cause some conflicts for this PR. Specifically, I'm planning on refiling a lot of things from symex-misc.el more appropriately into symex-traversals.el and a new symex-runtime.el (for things like evaluating expressions, looking up docs, etc.), and eliminating symex-misc.el. In case I get to this before your PR is merged, I think it shouldn't be too complicated to figure out where your changes would go as there isn't a significant overlap in that module, but in any case, I'm happy to help with any rebase or merge issues you might have, or you could potentially start with a fresh branch if that would be less annoying.

[devcarbon-com]

@countvajhula Thanks for the heads up!

I'm almost done with "marathon mode" that I've been in for my work project. I'll be back to wrap this up after that. (Tentative ETA next week).

[devcarbon-com]

Hi @countvajhula, my apologies for the radio silence.

I expect to have some time to work on this in the next month or two.

[countvajhula]

That's great to hear @devcarbon-com . I have had other work take priority in these last few months, but I hope to have time to dedicate to Symex soon as well, perhaps around the same time as you.