dimming error with org-agenda buffer when org-plus-contrib is installed

[gonewest818]

Reposting from email, where @mack1070101 wrote:

Using the customization from #32 (and probably without), if you have a "regular" buffer, and an org-agenda buffer split, if you select a different running application when you tab back to Emacs the agenda buffer will now always stay dimmed regardless if I toggle between splits. I've attached a video that's hopefully a little more clear.

[henmitch]

This also happens on my machine. I used a pared-down init.el (only packages were dimmer and org), and I got the same result. For what it's worth, this has been happening since before #32. It also happens even when I include (add-to-list 'dimmer-exclusion-regexp-list "^\\*Org Agenda\\*$") in my init.

Love the package! Thanks for all you're doing, and let me know if there's anything I can do.

[gonewest818]

I want to get to the bottom of this, but a possible workaround for now is to make dimmer stop dimming everything when you switch apps:

    (setq dimmer-watch-frame-focus-events nil)

And then, can you show me your pared-down init.el, and the steps you take to reproduce this? Does it happen every time you switch away from Emacs and back again?

[henmitch]

Using use-package for convenience:

(package-initialize)

(require 'package)

(setq package-enable-at-startup nil)

(use-package org
  :ensure t
  :config
  (setq org-agenda-files
    '("~/Todo/school/" "~/Todo/life")))

(use-package dimmer
  :ensure t
  :config
  (setq dimmer-fraction 0.4)
  (dimmer-mode))

All I need to do is open my agenda (M-x org-agenda a) and change to another window (by clicking or command-shift). All splits dim, but the agenda remains dimmed when I return to Emacs. The only way to un-dim it is to disable dimmer-mode (or quit and reopen Emacs). Even killing the agenda buffer and reopening it leaves it dimmed. I'll try that workaround!

[gonewest818]

Try as I might, I cannot get this to happen? I've tested on macos Catalina, Emacs 26.3 and Emacs 27.0.50.

Here's the init and org file I'm using (based on a template I use for testing packages in CircleCI). What's different about my init.el is that this one overrides the user-emacs-directory so that all the packages are downloaded fresh from melpa, and don't interfere with the ones you have in your personal config.

https://gist.github.com/gonewest818/a02fd215d9790acd837c873775e7329a

[henmitch]

I'm on Catalina (10.15.3), running Emacs 26.3. Using the init file you provided, the problem goes away. I'm not quite sure what that indicates (I'm a relative newbie), so what are my next steps? I've got the same versions of all the packages as were downloaded by your init.

[gonewest818]

Well, I'm not sure. That seems to mean something else in your ~/.emacs.d directory is causing this error.

If you're interested in debugging further, I would try a "binary search":

1. make a full copy of your ~/.emacs.d
2. put my init.el inside that folder
3. run the test (and confirm it fails)
4. start deleting stuff from that folder until you can can make the test succeed
5. the last thing you deleted that made the test succeed, is the thing that caused the problem

So in detail that might look like this

$ cd ~
$ cp -R .emacs.d test-emacs.d
$ cp gonewest-init.el test-emacs.d/init.el
$ emacs -q -l test-emacs.d/init.el
    # inside emacs, try opening org-agenda and then switch to another app

Based on what you're telling me this test will probably fail as you've reported.

Assuming it does fail, then this is where it gets interesting. Start deleting stuff from the test-emacs.d directory you just created. The first thing I would look at, are the various data files that packages have written into your .emacs.d. For me that would be things like the eshell history file, recentf, projectile cache, bookmark file, custom.el (if you have customized custom-file). Try deleting those things one at a time to see if any of those are causing this error.

If none of that helps, then you can start deleting packages in elpa:

$ cd test-emacs.d/elpa
$ rm -rf [a-m]*
    # in other words, you just deleted roughly half of the elpa directory
    # don't worry if you deleted dimmer or some other important dependency
    # because the init.el will install anything it needs
$ emacs -q -l test-emacs.d/init.el
    # inside emacs, try opening org-agenda and see what happens when you switch apps

If this test still fails, then maybe it's one of packages that still remains. Keep deleting another 50% of the packages until you find a package that seems to cause the problem.

[henmitch]

Looks like the issue is with org-plus-contrib (and verified). @mack1070101, do you find the same?

[gonewest818]

Good hunting! I’ll start digging into that.

[gonewest818]

Ok, I've got the reproducible case. I updated that gist with a new init.el that exhibits the problem. In my case it isn't necessary to lose/regain focus, by the way. Whenever I create an agenda with [C-c a n] that particular buffer is always dimmed.

Under the hood, I've been poking around with the debugging functions. (dimmer--debug-face-remapping-alist "*Org Agenda*") shows that there are TWO sets of "dimmed" remaps on every face in the *Org Agenda* buffer. Which means anytime you enter the buffer just one copy of the remaps is removed but the other copy stays around, and hence the buffer always appears dimmed. There's something about the way the *Agenda Commands* buffer and then the *Org Agenda* buffer are created/displayed/destroyed that is tripping up the functions that normally take care of the remaps.

[gonewest818]

I've narrowed it down to the version of org-agenda that you get with org-plus-contrib. Something changed between org 9.1.9 (in Emacs 26.3) and org 9.3.6 (in org-plus-contrib).

[gonewest818]

I'm about to push a fix for this.

The bug was that the *Org Agenda* buffer was getting dimmed, but the variable I use to track whether or not that buffer is dimmed was being reset in the middle of the creation process.

I learned something subtle about elisp while tracking this down, which is that buffer local variables can be reset to nil when a new major mode is applied to a buffer. The fix is only a single statement declaring the buffer local variable to be "permanent-local", i.e. can't be reset like that.

[mack1070101]

Thanks for fixing this @gonewest818! Looking forward to getting this fix