Problem calling evil-scroll-up interactively in ert test

[codygman]

Issue type

• Bug report

Environment

Emacs version: Operating System: NixOS 19.09.1254.9104be2ee08 Evil version: Evil version 1.2.14 Evil installation type: use-package Graphical/Terminal: X Tested in a make emacs session (see CONTRIBUTING.md): No

Reproduction steps

• Start Emacs

load-init-then-run-ert.el:

(require 'ert)
(load (expand-file-name "emacs-config.el"))
(load (expand-file-name "my-tests.el"))
(ert-run-tests-batch-and-exit)

emacs-config.el:

(require 'package)
(setq package-enable-at-startup nil)
(add-to-list 'package-archives '("melpa" . "http://melpa.org/packages/"))
(package-initialize)

;; Bootstrap `use-package'
(unless (package-installed-p 'use-package)
  (package-refresh-contents)
  (package-install 'use-package))

(use-package evil
  :ensure t
  :init
  (setq evil-want-integration t
    evil-want-keybinding nil
    evil-want-C-u-scroll t)
  :config
  (setq evil-symbol-word-search t)
  (evil-mode 1))

my-tests.el:

(defun my-emacs-everywhere-directory ()
  (if (eq nil (getenv "TRAVIS_OS_NAME"))
      "~/.emacs.d"
    "~/build/codygman/my-emacs-everywhere/"))

(defun open-lorem-ipsum-goto-end-scroll-up-return-char-position ()
  (save-excursion
    (find-file "testdata/loremipsum.txt")
    (goto-char (point-max))
    (call-interactively 'evil-scroll-up)
    (point)))

;; START evil
(ert-deftest evil-ctrl-u-scrolls-up ()
  (should (eq 4599 (open-lorem-ipsum-goto-end-scroll-up-return-char-position))))
;; END evil

Running it:

emacs -Q -batch --load load-init-then-run-ert.el

Expected behavior

In my ert test evil-scroll-up should scroll up and not throw a fixnum-p error.

Actual behavior

With an odd error:

Done (Total of 18 files compiled, 2 skipped)
ad-handle-definition: `evil-mode' got redefined
Loading /home/travis/build/codygman/my-emacs-everywhere/my-tests.el (source)...
Running 1 tests (2019-11-28 05:18:21+0000, selector `t')
Test evil-ctrl-u-scrolls-up backtrace:
  signal(wrong-type-argument (fixnump nil))
  apply(signal (wrong-type-argument (fixnump nil)))
  (setq value-2 (apply fn-0 args-1))
  (unwind-protect (setq value-2 (apply fn-0 args-1)) (setq form-descri
  (if (unwind-protect (setq value-2 (apply fn-0 args-1)) (setq form-de
  (let (form-description-4) (if (unwind-protect (setq value-2 (apply f
  (let ((value-2 'ert-form-evaluation-aborted-3)) (let (form-descripti
  (let* ((fn-0 #'eq) (args-1 (condition-case err (let ((signal-hook-fu
  (lambda nil (let* ((fn-0 #'eq) (args-1 (condition-case err (let ((si
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name evil-ctrl-u-scrolls-up :documentation
  ert-run-or-rerun-test(#s(ert--stats :selector t :tests [#s(ert-test 
  ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
  ert-run-tests-batch(nil)
  ert-run-tests-batch-and-exit()
  eval-buffer(#<buffer  *load*> nil "/home/travis/build/codygman/my-em
  load-with-code-conversion("/home/travis/build/codygman/my-emacs-ever
  load("/home/travis/build/codygman/my-emacs-everywhere/lo..." nil t)
  command-line-1(("--load" "load-init-then-run-ert.el"))
  command-line()
  normal-top-level()
Test evil-ctrl-u-scrolls-up condition:
    (wrong-type-argument fixnump nil)
   FAILED  1/1  evil-ctrl-u-scrolls-up (0.030792 sec)

Further notes

https://travis-ci.com/codygman/my-emacs-everywhere/jobs/261210520

[TheBB]

A simpler repro would definitely help here, preferably one that starts with make emacs and does not involve references to lorem ipsum files that I don't have.

I suspect that the problem is you're running in batch mode, and certain things that rely on the presence of windows do not work well in batch mode. There's a reason evil's test suite does NOT use --batch: the tests will predictably fail there. Is this issue reproducible in a regular graphical or TUI session?

[codygman]

It does seem to be batch mode that is the issue. I guess you guys somehow get success or failure status from running tests in emacs for your CI without batch mode? I'll go look at how that's done I guess.

https://github.com/codygman/evil/tree/emacs-interactive-evil-scroll-up-repro

I added a couple things to the makefile and this was the result:

;; start emacs with C-u scroll/want integration settings like I had
;; make emacs-repro
;; open bind-then-scroll-up-batch.el
;; split windows
;; run each line with eval-expression
;; no issue

;; try running the batch mode added to the makefile

;; make emacs-repro-batch
;; emacs -Q -L . -L lib -l goto-chg.el -l evil-tests.el \
;; --eval "(setq evil-want-integration t evil-want-keybinding nil evil-want-C-u-scroll t)" \
;; --eval "(evil-mode 1)" \
;; -batch -l bind-then-scroll-up-batch.el
;; Wrong type argument: fixnump, nil
;; make: *** [Makefile:80: emacs-repro-batch] Error 255

;; it does only happen in batch mode

[codygman]

Alright I see first you run make test then you have to write success/failure to a textfile in evil-tests-run.

I'll probably have issues with other packages that need windows as well so I might as well change my tests to work the same way for now. I guess it's hard for --batch mode to simulate windows functionality?

[codygman]

There's more to this and I'm trying to figure out how to make a minimal repro, but I just figured out I get this error with tests like evil does interactively if I call (kill-emacs) anywhere in the file.

[codygman]

I've added a reproduction and failing test with make test in my fork of evil and submitted a draft pull request linked above.

[codygman]

Per here looks like the error is happening because of:

$ cat trace-evil-scroll-up-results.txt

======================================================================

1 -> (evil-scroll-up nil)

1 <- evil-scroll-up: !non-local\ exit!

I don't know what non-local exit means, but I found Non-Local exits in the gnu emacs manual.

I know that unwind-protect does some low-level stuff to preserve lots of details back to how they were before you did stuff within that block.

I feel like (due to inconsistent instances of this test passing when I removed (kill-emacs)) it has something to do with killing emacs within an unwind-protect which likely uses some of that low level functionality and exiting within unwind-protect is something unexpected. That's just a guess though.

[TheBB]

Feel free to continue pursuing this if you wish. The testing suite has always been a bit fickle so I'm not surprised it's possible to break it with innocent-looking code, but as far as I can tell no actual bug in usage of evil is demonstrated.