Duplicated layer warnings

sinewalker commented 7 years ago

After upgrading to spacemacs 0.200.5 I now get warnings in the *spacemacs* buffer on startup:

                      Warnings:
                          - Duplicated layer squiz detected in directory
                            "/Users/mjl/.spacemacs.d/layers/squiz", replacing old directory
                            "/Users/mjl/.spacemacs.d/layers/squiz/" with new directory.      
                          - Duplicated layer org-journal detected in directory
                            "/Users/mjl/.spacemacs.d/layers/org-journal", replacing old directory
                            "/Users/mjl/.spacemacs.d/layers/org-journal/" with new directory.      
                          - Duplicated layer mjl-org detected in directory
                            "/Users/mjl/.spacemacs.d/layers/mjl-org", replacing old directory
                            "/Users/mjl/.spacemacs.d/layers/mjl-org/" with new directory.      
                          - Duplicated layer mjl detected in directory "/Users/mjl/.spacemacs.d/layers/mjl",
                            replacing old directory "/Users/mjl/.spacemacs.d/layers/mjl/" with new
                            directory.

I don't understand where these duplicates are. I suspect that my layer-list build-up code is disturbed or being disturbed by changes in how spacemacs is doing layer loading after the update. Here's *Messages*:

Loading /Users/mjl/.emacs.d/core/core-load-paths.el (source)...done
Loading /Users/mjl/.spacemacs.d/init.el (source)...done
Setting the font...
Open the quickhelp.
(Spacemacs) Warning: Duplicated layer squiz detected in directory "/Users/mjl/.spacemacs.d/layers/squiz", replacing old directory "/Users/mjl/.spacemacs.d/layers/squiz/" with new directory.
(Spacemacs) Warning: Duplicated layer org-journal detected in directory "/Users/mjl/.spacemacs.d/layers/org-journal", replacing old directory "/Users/mjl/.spacemacs.d/layers/org-journal/" with new directory.
(Spacemacs) Warning: Duplicated layer mjl-org detected in directory "/Users/mjl/.spacemacs.d/layers/mjl-org", replacing old directory "/Users/mjl/.spacemacs.d/layers/mjl-org/" with new directory.
(Spacemacs) Warning: Duplicated layer mjl detected in directory "/Users/mjl/.spacemacs.d/layers/mjl", replacing old directory "/Users/mjl/.spacemacs.d/layers/mjl/" with new directory.
Loading /Users/mjl/.emacs.d/layers/+completion/auto-completion/packages.el (source)...done
...

I think the core library is run after .emacs.d/core/layers/autolayer.el (or something like that in the minibuffer) -- it's been truncated or the logging level needs turning up?

sinewalker commented 7 years ago

so far as I can see, there's no duplicate in the dotspacemacs-configuration-layer-path:

~ λ ls -l $dotspacemacs-configuration-layer-path
total 1
drwxr-xr-x   7 mjl            staff    238 2016-11-04 22:41 mjl
drwxr-xr-x   6 mjl            staff    204 2016-04-19 21:57 mjl-org
drwxr-xr-x   4 mjl            staff    136 2016-07-01 23:24 org-journal
drwxr-xr-x   7 mjl            staff    238 2016-11-02 15:01 squiz
~ λ 
~ λ echo $dotspacemacs-configuration-layer-path
~/.spacemacs.d/layers/
~ λ

(turned off company-mode so that can verify I'm seeing the right directory)

Also, the layers variable (and mjl--layers, which I built up) has no duplicate:

~ λ echo $mjl--layers
auto-completion :variables auto-completion-private-snippets-directory ~/.spacemacs.d/snippets/ better-defaults emacs-lisp git markdown org shell :variables shell-default-height 66 shell-default-position quote top spell-checking syntax-checking version-control colors php python javascript erc xkcd mjl-org deft puppet lua vagrant emoji org-journal osx mjl :variables mjl-bind-osx-keys mjl-bind-unix-keys clojure :variables clojure-enable-fancify-symbols t squiz :variables squiz-wiid-script expand-file-name ~/Work/lab/whyisitdown/whyisitdown squiz-wiid-keys expand-file-name ~/Work/lab/whyisitdown/keys mu4e :variables mu4e-installation-path /usr/local/Cellar/mu/0.9.16/share/emacs/site-lisp/mu/mu4e

~ λ echo $dotspacemacs-configuration-layers
auto-completion :variables auto-completion-private-snippets-directory ~/.spacemacs.d/snippets/ better-defaults emacs-lisp git markdown org shell :variables shell-default-height 66 shell-default-position quote top spell-checking syntax-checking version-control colors php python javascript erc xkcd mjl-org deft puppet lua vagrant emoji org-journal osx mjl :variables mjl-bind-osx-keys mjl-bind-unix-keys clojure :variables clojure-enable-fancify-symbols t squiz :variables squiz-wiid-script expand-file-name ~/Work/lab/whyisitdown/whyisitdown squiz-wiid-keys expand-file-name ~/Work/lab/whyisitdown/keys mu4e :variables mu4e-installation-path /usr/local/Cellar/mu/0.9.16/share/emacs/site-lisp/mu/mu4e
~ λ

sinewalker commented 7 years ago

process of elimination...

It's not a layers cache or something in .emacs.d: if I clone a fresh spacemacs with a completely new .emacs.d then it still has these warnings
It's not the layers in my layers path themselves: if I make a completly fresh ~/.spacemacs and then set dotspacemacs-configuration-layer-path '("~/.spacemacs.d.keep/layers") there are no warnings (and I verified the variable is set after loading)

So it must be something in my init.el. I also notice that org-mode is not hiding asterisks like it should (but it is in a brad new setup). So I suspect that when I merged in changes to the init template after updating spacemacs, I may have not closed a paren, or something.

sinewalker commented 7 years ago

Moving the layers to another location resolves.

As workaround commit 11cfd18 moves the entire ~/.spacemacs.d into ~/lib/spacemacs until I can figure out what's happening. Installation will require a symlink ~/.spacemacs -> ~/lib/spacemacs/init.el which is not as clean as I'd like, plus it SHOULD work how I had originally designed...

sinewalker commented 7 years ago

Playing with this again after updating Packages. Still getting the same Warnings if I move back to ~/.spacemacs.d

I just noticed: the "dupicates differ by having a trailing '/' or not.

This is interesting (from the core layer code):

(defun configuration-layer/discover-layers ()
  "Initialize `configuration-layer--indexed-layers' with layer directories."
  ;; load private layers at the end on purpose we asume that the user layers
  ;; must have the final word on configuration choices. Let
  ;; `dotspacemacs-directory' override the private directory if it exists.
  (setq  configuration-layer--indexed-layers (make-hash-table :size 1024))
  (let ((search-paths (append (list configuration-layer-directory)
                              dotspacemacs-configuration-layer-path
                              (list configuration-layer-private-layer-directory)
                              (when dotspacemacs-directory
                                (list dotspacemacs-directory))))

It includes the dotspacemacs-directory by default?

sinewalker commented 7 years ago

commenting out the ;dotspacemacs-configuration-layer-path in init.el doesn't help, but the layers are still found, and still duplicates.

sinewalker commented 7 years ago

The following patch to core/core-configuration-layer.el fixes:

modified   core/core-configuration-layer.el
@@ -1091,7 +1091,7 @@ Returns nil if the directory is not a category."
                   (if indexed-layer
                       ;; the same layer may have been discovered twice,
                       ;; in which case we don't need a warning
-                      (unless (string-equal (oref indexed-layer :dir) sub)
+                      (unless (string-equal (oref indexed-layer :dir) (concat sub "/"))
                         (configuration-layer//warning
                          (concat
                           "Duplicated layer %s detected in directory \"%s\", "

But there are two issues here:

This would mean patching spacemacs. I'm undecided if this is actually a bug in spacemacs.
I'm uncertain whether the trailing "/" should always be appended for comparison?

sinewalker commented 7 years ago

It is a bug, but fixed in Spacemacs Develop branch:

https://gitter.im/syl20bnr/spacemacs?at=584e83500da034021b9c684b

Michael Lockhart @sinewalker 21:45 hey there. I'm tracking an issue with my own dotspacemacs configuration, and I think I've found a problem with the core layers loader (or not, I'm unsure) sinewalker/dotspacemacs#1 I'm just asking around here for some tips/guidance before I file an issue in github. (I'm more than happy if someone can show me my mistake :wink: ) What happens is: I have custom layers in my ~/.spacemacs.d/layers and there are being picked up twice, once with a trailing slash, and once without. If I move the layers out of .spacemacs.d and change the layers path, there's no warning. If instead I change the spacemacs code to add a slash, there's also no warning ... SteveJobzniak @SteveJobzniak 22:00 @sinewalker That's a bug they fixed in the develop version of Spacemacs. As far as I know, there is no danger to the warning in master and the layer works anyway, so if you can live with the warning text at startup, just let it be for now. Michael Lockhart @sinewalker 22:01 wow! Yes I can live with it. I was only looking harder because I thought maybe I was Doing It Wrong, and it only came up after the upgrade to 0.2.something :grinning: thankyou @SteveJobzniak SteveJobzniak @SteveJobzniak 22:02 I agree, it's a confusing bug :grinning: No problem

sinewalker commented 7 years ago

https://github.com/syl20bnr/spacemacs/issues/7481

sinewalker commented 7 years ago

Closing: I can live with this knowing:

It's not causing any actual issue, just Warnings in the *spacemacs* buffer on start-up
It's been fixed in Spacemacs Develop, issue 7481

sinewalker commented 7 years ago

FYI, the PR for the fix

https://github.com/syl20bnr/spacemacs/pull/7482/commits/092810741f8b836ef536e89b9a1f97fece012300

Very similar, only he correctly uses directory-file-name instead of a dumb concat.

This comment from TheBB is interesting too:

I don't understand why the layer system checks these directories multiple times, but the PR looks all right enough. Thanks! Squashed and cherry-picked in develop. You can safely delete your branch.

Maybe the startup could be sped up further in the future?

ghost commented 7 years ago

Maybe the startup could be sped up further in the future?

Wait until you find out that theme switching with "SPC T n" (for "next theme") applies each theme twice, causing all syntax highlighters and theme code to apply themselves twice. Maybe theme switching could feel less sluggish if that's fixed. ;-)

Edit: Nah, I tried manually switching theme via the function to only apply it once, but the speed difference wasn't noticeable. Oh well.

sinewalker commented 7 years ago

Commit 6009600 reverts commit 11cfd18 : no longer required.

syl20bnr commented 7 years ago

I made some cleanup in commit: https://github.com/syl20bnr/spacemacs/commit/f5957eb

Should fix the multiple scan of same directories. No real improvement on loading time though.

sinewalker commented 7 years ago

I've switched to the Spacemacs develop branch in commit 6ec3706 which removes these warnings

sinewalker / dotspacemacs

Duplicated layer warnings #1