search

radian-software / straight.el

🍀 Next-generation, purely functional package manager for the Emacs hacker.
MIT License
2.69k stars 152 forks source link

straight is rebuilding org on every single launch of emacs #624

Closed aspiers closed 3 years ago

aspiers commented 3 years ago

What's wrong

I am now seeing a Building org... message from straight.el every single time I launch emacs. Presumably this is related to the recent addition of support for :build steps. Since the build steps do things like compiling the whole manual, this takes something like one or two minutes and therefore drastically increases the waiting time before emacs is ready for use.

Directions to reproduce

I've added some debug to the bit of straight-use-package which calculates modified and it is showing that (straight--package-might-be-modified-p recipe no-build) returns 98, so presumably it's something in there which is going wrong.

But if you can't reproduce this and it's looking like this might be related to some peculiarity of my config, given that I have a decent amount of elisp hacking experience, rather than trying to create a minimal test case I guess it might be quicker if you can confirm I'm on the right track and I'll keep digging in straight--package-might-be-modified-p.

Version information

progfolio commented 3 years ago

Try running the following test case twice. It will build straight/Org in a clean environment in /tmp/org-rebuild-test.straight on the first execution. Once that finishes, you should see something similar to the first Output below. Then run it again. It should run much faster (hopefully Org is not being rebuilt) the second time and the Output should be similar to the second Output block below.

Test Case ```emacs-lisp (straight-bug-report :user-dir "org-rebuild-test.straight" :preserve t :post-bootstrap (straight-use-package 'org)) ```
Output ```emacs-lisp Bootstrapping straight.el... Bootstrapping straight.el...done Rebuilding all packages due to build cache schema change Looking for gnu-elpa-mirror recipe → Cloning melpa... Looking for gnu-elpa-mirror recipe → Cloning melpa...done Looking for emacsmirror-mirror recipe → Cloning gnu-elpa-mirror... Looking for emacsmirror-mirror recipe → Cloning gnu-elpa-mirror...done Looking for straight recipe → Cloning emacsmirror-mirror... Looking for straight recipe → Cloning emacsmirror-mirror...done Building straight... Building straight...done Test run with version: prerelease (HEAD -> develop, origin/develop) 68dade0 2020-10-24 Cloning org... Cloning org...done Building org... Building org...done ```
Output<2> ```emacs-lisp Test run with version: prerelease (HEAD -> develop, origin/develop) 68dade0 2020-10-24 ```
aspiers commented 3 years ago

Thanks a lot for the pointers! I'm unsure if you meant to run within an existing emacs or a fresh emacs -Q or something else? If I just eval-defun that sexp then I only see:

Testing straight.el in directory: /tmp/org-rebuild-test.straight

and nothing else happens.

OTOH if you meant with emacs -Q or similar then I am unsure how the bootstrapping is supposed to work because of course straight-bug-report is not defined yet.

progfolio commented 3 years ago

Sorry, I should have been clearer. You'll want to run that in a regular Emacs session. It will start another Emacs process that runs the test. When it finishes, a buffer with the test's output (*straight-bug-report-process*) should pop up with the results. The test will install straight and Org, so it should take about as long as it does on your normal system.

progfolio commented 3 years ago

If you're not seeing anything after a reasonable amount of time, you can add to the test to make it an interactive emacs process:

(straight-bug-report
  :interactive t
  :user-dir "org-rebuild-test.straight"
  :preserve t
  :post-bootstrap 
  (straight-use-package 'org))

You can also check to see if /tmp/org-rebuild-test.straight exists on disk.

aspiers commented 3 years ago

@progfolio commented on November 2, 2020 7:37 PM:

Sorry, I should have been clearer.
You'll want to run that in a regular Emacs session.

That's what I did. It just outputs the message above in the minibuffer and nothing else happens... Ahah, eventually I saw this in the minibuffer:

error in process sentinel: unless: Symbol’s value as variable is void: interactive

It happened so much later (minutes later) that I didn't realise it was related. But running it again showed the connection. After enabling debug-on-error I get this:

Debugger entered--Lisp error: (void-variable interactive)
  (if interactive nil (unless raw (straight-bug-report--format report)) (switch-to-buffer-other-window straight-bug-report--process-buffer))
  (unless interactive (unless raw (straight-bug-report--format report)) (switch-to-buffer-other-window straight-bug-report--process-buffer))
  (lambda (_process _event) (unless interactive (unless raw (straight-bug-report--format report)) (switch-to-buffer-other-window straight-bug-report--process-buffer)) (unless preserve-files (when (file-exists-p temp-emacs-dir) (delete-directory temp-emacs-dir 'recursive))))(#<process *straight-bug-report-process*> "finished\n")
progfolio commented 3 years ago

Sounds like you may be executing it without lexical binding enabled. I patched this last night so that you'll get a more informative, immediate, user-error in this case. Try evaling in a buffer which has lexical-binding enabled.

aspiers commented 3 years ago

Ahah yes! Works now, many thanks :-) Yes, on second run it doesn't rebuild org. So does that mean the issue is specific to my environment? If so, what's the next step - my original suggestion of digging deeper in straight--package-might-be-modified-p?

progfolio commented 3 years ago

So does that mean the issue is specific to my environment?

Yes, that's my inclination. Test case runs for me without subsequent rebuilds. Stepping through straight--package-might-be-modified-p could be a good lead, but it might just be a case of another package marking buffers modified, too.

Some other useful info to have:

aspiers commented 3 years ago

Thanks a lot - super busy at the moment but I'll get back to this soon!

rileyrg commented 3 years ago

I have been having similar issues with org-plus-contrib. I have resorted to delete all content in straight/ but it still crops up. I'll follow the suggestions here too and see what happens. Interestingly doing a manual eval from withing my init file (C-x C-e outside use-package form) stopped this for a while. It's very annoying and very confusing ;) Screenshot from 2020-11-08 10-32-51

progfolio commented 3 years ago

@rileyrg please try running the test case here in a buffer with lexical binding enabled:

https://github.com/raxod502/straight.el/issues/624#issuecomment-720153249

Any other relevant info, such as your org use-package declaration and the values I mentioned here:

https://github.com/raxod502/straight.el/issues/624#issuecomment-720750877

will be helpful as well.

progfolio commented 3 years ago

@aspiers I noticed you are using use-package's :ensure keyword alongside the :straight keyword:

https://github.com/aspiers/emacs/blob/9eeee3e99e5e9129bb63a16e7fe81452bd0025bc/.emacs.d/init.d/as-org-mode.el#L8

This could cause a conflict w straight since :ensure utilizes package.el. Does removing that help?

aspiers commented 3 years ago

Nice idea but it doesn't help. However @rileyrg's screenshot above made me wonder why I was seeing Building org... rather than Building org-plus-contrib... so I looked closer and actually saw both in the *Messages* buffer, but the former is the one which takes a long time.

This suggests to me that maybe this issue is related to the way I'm loading org-plus-contrib.

aspiers commented 3 years ago

Hrm, I changed some stuff and now both builds are taking a long time ... :-/

If I have

(use-package org
  :straight org-plus-contrib
  ...)

does that mean that I should be referencing org or org-plus-contrib in clauses such as this :after section?

(use-package orgit
  :after (org-plus-contrib magit))
aspiers commented 3 years ago

@progfolio commented on November 2, 2020 10:06 PM:

Some other useful info to have:

  • value of straight-check-for-modifications

The default: (find-at-startup find-when-checking) (incidentally, checking this led to #628)

  • recipe for Org in your init file and recipe for Org as returned by straight-get-recipe.

org or org-plus-contrib? In any case, here they both are:

`(org :type git :repo "https://code.orgmode.org/bzg/org-mode.git" :local-repo "org" :build ,(let ((make (if (eq system-type 'berkeley-unix) "gmake" "make")) (emacs (concat "EMACS=" invocation-directory invocation-name))) `(,make "oldorg" ,emacs)))

`(org-plus-contrib :type git :repo "https://code.orgmode.org/bzg/org-mode.git" :local-repo "org" :files (:defaults "contrib/lisp/*.el") :build ,(let ((make (if (eq system-type 'berkeley-unix) "gmake" "make")) (emacs (concat "EMACS=" invocation-directory invocation-name))) `(,make "oldorg" ,emacs)))
  • output of (gethash "org-plus-contrib" straight--build-cache) (or "org" if you're not using "org-plus-contrib")
("2020-11-08 11:21:04" nil (:type git :repo "https://code.orgmode.org/bzg/org-mode.git" :local-repo "org" :files (:defaults "contrib/lisp/*.el") :build ("make" "oldorg" "EMACS=/usr/bin/emacs") :package "org-plus-contrib"))
aspiers commented 3 years ago

@aspiers commented on November 8, 2020 11:19 AM:

Hrm, I changed some stuff and now both builds are taking a long time ... :-/

If I have

(use-package org
  :straight org-plus-contrib
  ...)

does that mean that I should be referencing org or org-plus-contrib in clauses such as this :after section?

(use-package orgit
  :after (org-plus-contrib magit))

Oh wow, I changed all of these from org-plus-contrib to org and now the constant rebuilding is gone!

aspiers commented 3 years ago

I guess there are two remaining questions:

  1. @rileyrg does this help you fix it?
  2. Are there any improvements which could be made to the docs in this regard?
rileyrg commented 3 years ago

I'm also having constant issues with boxquote.el btw. Keeps saying can't run git. I'm only on my mobile atm so just consider this as a "you're not alone" support. The strangest thing to me re org-contrib is that C-x C-e on the use-package builds it and then it's not rebuilt on subsequent restarts as we'd hope.

On Sun, 8 Nov 2020, 12:44 Adam Spiers notifications@github.com wrote:

@progfolio https://github.com/progfolio commented on November 2, 2020 10:06 PM https://github.com/raxod502/straight.el/issues/624#issuecomment-720750877 :

Some other useful info to have:

  • value of straight-check-for-modifications

The default: (find-at-startup find-when-checking)

  • recipe for Org in your init file and recipe for Org as returned by straight-get-recipe.

org or org-plus-contrib? In any case, here they both are:

(org :type git :repo "https://code.orgmode.org/bzg/org-mode.git" :local-repo "org" :build ,(let ((make (if (eq system-type 'berkeley-unix) "gmake" "make")) (emacs (concat "EMACS=" invocation-directory invocation-name)))(,make "oldorg" ,emacs)))

(org-plus-contrib :type git :repo "https://code.orgmode.org/bzg/org-mode.git" :local-repo "org" :files (:defaults "contrib/lisp/*.el") :build ,(let ((make (if (eq system-type 'berkeley-unix) "gmake" "make")) (emacs (concat "EMACS=" invocation-directory invocation-name)))(,make "oldorg" ,emacs)))

  • output of (gethash "org-plus-contrib" straight--build-cache) (or "org" if you're not using "org-plus-contrib")

("2020-11-08 11:21:04" nil (:type git :repo "https://code.orgmode.org/bzg/org-mode.git" :local-repo "org" :files (:defaults "contrib/lisp/*.el") :build ("make" "oldorg" "EMACS=/usr/bin/emacs") :package "org-plus-contrib"))

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/raxod502/straight.el/issues/624#issuecomment-723566007, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACYTC6R3CLGSQB7PF34JMDSO2ACTANCNFSM4TGXVASQ .

aspiers commented 3 years ago

This issue describes the problem where it is rebuilt, undesirably, on subsequent restarts. Are you saying you want the opposite behaviour? It's not supposed to rebuild unless something changes which requires the rebuild.

rileyrg commented 3 years ago

This issue describes the problem where it is rebuilt, undesirably, on subsequent restarts. Are you saying you want the opposite behaviour? It's not supposed to rebuild unless something changes which requires the rebuild.

I am saying that it was constantly rebuilding up to when I started emacs again, watched it rebuild org-plus-contrib yet again, opened my config file and manually eval'ed the org use-package form. Then, when I quit and restarted emacs it didn't rebuild. Strange eh? But it might trigger an "aha" in someone who knows this stuff better than me.

My org-mode use package

rileyrg commented 3 years ago

FYI, I have to give up now because of other things today, but I did a complete straight wipe out and restarted emacs. I had to comment out the boxquote.el form. 

     (use-package boxquote
       :bind
       ("C-S-r" . boxquote-region))

When I tried to evaluate it after here is the stack. I've a nasty feeling this is going to be something to do with the recipe format but I've no idea. All the other GitHub hosted repos clone just fine.

`Debugger entered--Lisp error: (error "Failed to run \"git\"; see buffer straight-process") signal(error ("Failed to run \"git\"; see buffer straight-process")) error("Failed to run %S; see buffer %s" "git" "straight-process") straight--get-call-raw("git" "clone" "--origin" "origin" "--branch" "master" "--no-checkout" "git@github.com:davep/boxquote.el.git" "/home/rgr/.emacs.d/straight/repos/boxquote.el/") apply(straight--get-call-raw "git" ("clone" "--origin" "origin" "--branch" "master" "--no-checkout" "git@github.com:davep/boxquote.el.git" "/home/rgr/.emacs.d/straight/repos/boxquote.el/")) straight--get-call("git" "clone" "--origin" "origin" "--branch" "master" "--no-checkout" "git@github.com:davep/boxquote.el.git" "/home/rgr/.emacs.d/straight/repos/boxquote.el/") straight-vc-git--clone-internal(:depth full :remote "origin" :url "git@github.com:davep/boxquote.el.git" :repo-dir "/home/rgr/.emacs.d/straight/repos/boxquote.el/" :branch "master" :commit nil) straight-vc-git-clone((:type git :flavor melpa :host github :repo "davep/boxquote.el" :package "boxquote" :local-repo "boxquote.el") nil) apply(straight-vc-git-clone ((:type git :flavor melpa :host github :repo "davep/boxquote.el" :package "boxquote" :local-repo "boxquote.el") nil)) straight-vc(clone git (:type git :flavor melpa :host github :repo "davep/boxquote.el" :package "boxquote" :local-repo "boxquote.el") nil) straight-vc-clone((:type git :flavor melpa :host github :repo "davep/boxquote.el" :package "boxquote" :local-repo "boxquote.el")) straight--clone-repository((:type git :flavor melpa :host github :repo "davep/boxquote.el" :package "boxquote" :local-repo "boxquote.el") nil)

f(compiled-function () #<bytecode 0x157cc17d2bb5>)()

straight--transaction-exec(use-package-(:type\ git\ :flavor\ melpa\ :host\ github\ :repo\ \"davep/boxquote.el\"\ :package\ \"boxquote\"\ :local-repo\ \"boxquote.el\")-nil-nil :now #f(compiled-function () #<bytecode 0x157cc17d2bb5>)) straight-use-package(boxquote) (progn (straight-use-package 'boxquote) (defvar use-package--warning101 #'(lambda (keyword err) (let ((msg (format "%s/%s: %s" ... keyword ...))) (display-warning 'use-package msg :error)))) (condition-case err (progn (if (fboundp 'boxquote-region) nil (autoload #'boxquote-region "boxquote" nil t)) (let ((name "C-S-r") (key (if (vectorp name) name (read-kbd-macro name))) (kmap (if (and nil ...) (symbol-value nil) nil)) (kdesc (cons (if ... name ...) (if ... nil ...))) (binding (lookup-key (or kmap global-map) key))) (let ((entry (assoc kdesc personal-keybindings)) (details (list ... ...))) (if entry (setcdr entry details) (add-to-list 'personal-keybindings (cons kdesc details)))) (define-key (or kmap global-map) key #'boxquote-region))) ((debug error) (funcall use-package--warning101 :catch err)))) eval((progn (straight-use-package 'boxquote) (defvar use-package--warning101 #'(lambda (keyword err) (let ((msg ...)) (display-warning 'use-package msg :error)))) (condition-case err (progn (if (fboundp 'boxquote-region) nil (autoload #'boxquote-region "boxquote" nil t)) (let ((name "C-S-r") (key (if ... name ...)) (kmap (if ... ... nil)) (kdesc (cons ... ...)) (binding (lookup-key ... key))) (let ((entry ...) (details ...)) (if entry (setcdr entry details) (add-to-list ... ...))) (define-key (or kmap global-map) key #'boxquote-region))) ((debug error) (funcall use-package--warning101 :catch err)))) nil) elisp--eval-last-sexp(nil) eval-last-sexp(nil) funcall-interactively(eval-last-sexp nil) call-interactively(eval-last-sexp nil nil) command-execute(eval-last-sexp) `

aspiers commented 3 years ago

That error simply means the git clone failed, which could be down to a broken internet connection or some other misconfiguration of your git setup. I suggest manually trying the git clone command listed in the error to see if it's even related to emacs.

rileyrg commented 3 years ago

The clone works fine converting to

git clone https://github.com/davep/boxquote.el

indicating it's something (recent) with my ssh/gnupg agent in this case. I'll change the recipe. The only one in the hundred or so packages I use :(

On Sun, 8 Nov 2020, 14:31 Adam Spiers notifications@github.com wrote:

That error simply means the git clone failed, which could be down to a broken internet connection or some other misconfiguration of your git setup. I suggest manually trying the git clone command listed in the error to see if it's even related to emacs.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/raxod502/straight.el/issues/624#issuecomment-723577739, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACYTC2LNSAQD26CNLUNXOTSO2MT3ANCNFSM4TGXVASQ .

rileyrg commented 3 years ago

I just did a clean start ensuring emacs was opening my key properly for ssh-agent. the git@github boxquote.el clone still failed from inside emacs. I'm removing it now so I can get on. Sorry I cant provide more info atm.

progfolio commented 3 years ago

@rileyrg: I get the git failure as well. The error being thrown points to the *straight-process* buffer which reveals:

Cloning into '/home/n/.emacs.d/straight/repos/boxquote.el'...
fatal: Remote branch master not found in upstream origin

It appears boxquote.el's default branch is now "main":

https://github.com/davep/boxquote.el

either of these should fix the problem while we wait for #543 to land:

(straight-use-package '(boxquote :branch "main"))

or

(use-package boxquote
  :straight (:branch "main"))
progfolio commented 3 years ago

@aspiers: Thank you for taking the time to look into this more and for #628. Glad to see you were able to resolve the issue. I agree we could do better by providing more specific configuration examples in the docs for popular packages like Org. If you have ideas, please let us know.

I think the changes planned for Org 9.5 (#620) may help alleviate some of this confusion.

aspiers commented 3 years ago

Urgh, actually I'm not sure I have resolved this. The problem with changing them all from org-plus-contrib to org is that I think straight is actually now building both, but presumably I only want it to build org-plus-contrib. I agree that #620 will help by preventing the risk of this duplication, but it won't necessarily solve any underlying problem with unnecessary rebuilding.

aspiers commented 3 years ago

Here's a test I used:

(use-package org-bullets
  :straight t
  :after org-plus-contrib    ;; N.B. *not* just org here
  :config
  (require 'org-contacts)

(use-package org
  :straight org-plus-contrib)

org-contacts.el is only in Org's contrib/lisp so this tests whether org-plus-contrib is being correctly loaded. If the :after is changed to org or commented out, the require will fail.

progfolio commented 3 years ago

Does moving the use-package declaration for org before the declaration for org-bullets help?

aspiers commented 3 years ago

Yes but the whole point of using :after was to avoid dependencies on lexical ordering as my emacs config is split across many files.

progfolio commented 3 years ago

I understand. I just wanted to make sure we were at the root cause of the issue. The semantics of use-package's :after keyword only affect loading. Even if we take straight out of the equation and just :ensure a package, it will still be installed prior to it being loaded by an :after declaration:

(use-package a
  :ensure t
  :after b)

expands to:

(progn
  (use-package-ensure-elpa 'a
                           '(t)
                           'nil)
  (eval-after-load 'b
    '(require 'a nil nil)))

Similarly, with :straight t (or any variation) the expansion is:

(progn
  (straight-use-package 'a)
  (eval-after-load 'b
    '(require 'a nil nil)))

You could install org-plus-contrib earlier in your init and then configure it in a separate declaration/file by either having two use-package forms, or by directly calling (straight-use-package 'org-plus-contrib) before loading packages which transitively load Org. I do something similar to the latter in my config. I agree it's not as elegant as one would hope and am open to a cleaner solution if you can think of one.

rileyrg commented 3 years ago

Sorry chaps, I've been out of it. This main/master solution I'll try later today. Thanks. It seems progress bring made

On Tue, 10 Nov 2020, 05:24 Nicholas Vollmer notifications@github.com wrote:

I understand. I just wanted to make sure we were at the root cause of the issue. The semantics of use-package's :after keyword only affect loading. Even if we take straight out of the equation and just :ensure a package, it will still be installed prior to it being loaded by an :after declaration:

(use-package a :ensure t :after b)

expands to:

(progn (use-package-ensure-elpa 'a '(t) 'nil) (eval-after-load 'b '(require 'a nil nil)))

Similarly, with :straight t (or any variation) the expansion is:

(progn (straight-use-package 'a) (eval-after-load 'b '(require 'a nil nil)))

You could install org-plus-contrib earlier in your init and then configure it in a separate declaration/file by either having two use-package forms, or by directly calling (straight-use-package 'org-plus-contrib) before installing packages which would transitively load Org. I do something similar to the latter in my config.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/raxod502/straight.el/issues/624#issuecomment-724441252, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACYTC6H2KDR2TCLWXLSQ2TSPC57LANCNFSM4TGXVASQ .

rileyrg commented 3 years ago

@rileyrg please try running the test case here in a buffer with lexical binding enabled:

#624 (comment)

Any other relevant info, such as your org use-package declaration and the values I mentioned here:

#624 (comment)

will be helpful as well.

I

(use-package org
        :straight org-plus-contrib
        :custom
        (org-directory "~/.emacs.d/orgfiles")
        (org-agenda-files (list org-directory (concat org-directory "/projects" )))
        (org-refile-targets `((,(directory-files-recursively org-directory "^[[:alnum:]].*\\.\\(org\\|gpg\\)\\'") :maxlevel . 9)))
        (org-outline-path-complete-in-steps nil)         ; Refile in a single go
        (org-refile-use-outline-path t)                  ; Show full paths for refiling
        :config
        (require 'org-protocol)
        (require 'org-inlinetask)
        (defun rgr/org-agenda (&optional arg)
          (interactive "P")
          (let ((org-agenda-tag-filter-preset '("-trash")))
            (org-agenda arg "a")))
        :bind
        ("C-c a" . org-agenda)
        ("C-c A" . rgr/org-agenda)
        ("C-c c" . org-capture)
        ("C-c l" . org-store-link)
        ("C-c C-l" . org-insert-link)
        ("C-c C-s" . org-schedule)
        ("C-c t" . org-todo))

Output:-

Test Case ```emacs-lisp (straight-bug-report :user-dir "org-rebuild-test.straight" :preserve t :post-bootstrap (straight-use-package 'org)) ```
Output ```emacs-lisp Bootstrapping straight.el... Bootstrapping straight.el...done Rebuilding all packages due to build cache schema change Looking for gnu-elpa-mirror recipe → Cloning melpa... Looking for gnu-elpa-mirror recipe → Cloning melpa...done Looking for emacsmirror-mirror recipe → Cloning gnu-elpa-mirror... Looking for emacsmirror-mirror recipe → Cloning gnu-elpa-mirror...done Looking for straight recipe → Cloning emacsmirror-mirror... Looking for straight recipe → Cloning emacsmirror-mirror...done Building straight... Building straight...done Test run with version: prerelease (HEAD -> develop, origin/develop) 0f283e2 2020-11-04 Cloning org... Cloning org...done Building org... Building org...done ```

output of (gethash "org-plus-contrib" straight--build-cache):-

("2020-11-10 10:13:43" nil (:type git :repo "https://code.orgmode.org/bzg/org-mode.git" :local-repo "org" :files (:defaults "contrib/lisp/*.el") :build ("make" "oldorg" "EMACS=/usr/local/bin/emacs") :package "org-plus-contrib"))

straight-check-for-modifications is a variable defined in straight.el.

Value (find-at-startup find-when-checking)

Original Value (find-at-startup find-when-checking)

progfolio commented 3 years ago

@rileyrg Thanks for running that. Try running it again and sharing the output. (If you've restarted since the first run or emptied your temporary file directory, you will have to run it two more times). That way we can see if Org is being built on a subsequent startup or if it is a matter of configuration.

rileyrg commented 3 years ago

@rileyrg Thanks for running that. Try running it again and sharing the output. (If you've restarted since the first run or emptied your temporary file directory, you will have to run it two more times). That way we can see if Org is being built on a subsequent startup or if it is a matter of configuration.

Two straight runs (pun intended ;)):-

Test Case ```emacs-lisp (straight-bug-report :user-dir "org-rebuild-test.straight" :preserve t :post-bootstrap (straight-use-package 'org)) ```
Output ```emacs-lisp Bootstrapping straight.el... Bootstrapping straight.el...done Rebuilding all packages due to build cache schema change Looking for gnu-elpa-mirror recipe → Cloning melpa... Looking for gnu-elpa-mirror recipe → Cloning melpa...done Looking for emacsmirror-mirror recipe → Cloning gnu-elpa-mirror... Looking for emacsmirror-mirror recipe → Cloning gnu-elpa-mirror...done Looking for straight recipe → Cloning emacsmirror-mirror... Looking for straight recipe → Cloning emacsmirror-mirror...done Building straight... Building straight...done Test run with version: prerelease (HEAD -> develop, origin/develop) f7ff754 2020-11-11 Cloning org... Cloning org...done Building org... Building org...done ```
Test Case ```emacs-lisp (straight-bug-report :user-dir "org-rebuild-test.straight" :preserve t :post-bootstrap (straight-use-package 'org)) ```
Output ```emacs-lisp Test run with version: prerelease (HEAD -> develop, origin/develop) f7ff754 2020-11-11 ```

I should mention though that hand evaling my org form AFER emacs has fully loaded and rebuilt org-plus-contrib solves the issue for subsequent restarts too.

progfolio commented 3 years ago

@rileyrg :

Looks like the test only builds Org on the first start. You can add this to the start of your config:

  (eval-after-load "org" '(debug))

and launch emacs with emacs --debug-init. You should hit the debugger immediately after Org is loaded. It would be useful to see when/where it is being loaded.

Did adding the :branch to the boxquote recipe fix the other issue for you?

rileyrg commented 3 years ago

@rileyrg :

Looks like the test only builds Org on the first start. You can add this to the start of your config:

  (eval-after-load "org" '(debug))

and launch emacs with emacs --debug-init. You should hit the debugger immediately after Org is loaded. It would be useful to see when/where it is being loaded.

Debugger entered: nil (lambda nil (debug))() funcall((lambda nil (debug))) mapc(funcall ((lambda nil (debug)))) do-after-load-evaluation("/home/rgr/.emacs.d/straight/build/org-plus-contrib...") load-with-code-conversion("/home/rgr/.emacs.d/straight/build/org-plus-contrib..." "/home/rgr/.emacs.d/straight/build/org-plus-contrib..." nil t) require(org) apply(require org) byte-compile-file-form-require((require 'org)) byte-compile-file-form((require 'org))

f(compiled-function (form) #<bytecode 0x15613a213919>)((require 'org))

byte-compile-recurse-toplevel((require 'org) #f(compiled-function (form) #<bytecode 0x15613a213919>)) byte-compile-toplevel-file-form((require 'org))

f(compiled-function (inbuffer) #<bytecode 0x15613a2ee365>)(#<buffer Compiler Input>)

byte-compile-from-buffer(#<buffer Compiler Input>) byte-compile-file("/home/rgr/.emacs.d/straight/build/org-plus-contrib..." nil) byte-recompile-file("/home/rgr/.emacs.d/straight/build/org-plus-contrib..." force 0)

f(compiled-function () #<bytecode 0x15613a551f9d>)()

byte-recompile-directory("/home/rgr/.emacs.d/straight/build/org-plus-contrib..." 0 force) straight--byte-compile-package((:type git :repo "https://code.orgmode.org/bzg/org-mode.git" :local-repo "org" :files (:defaults "contrib/lisp/.el") :build ("make" "oldorg" "EMACS=/usr/local/bin/emacs") :package "org-plus-contrib")) straight--build-package((:type git :repo "https://code.orgmode.org/bzg/org-mode.git" :local-repo "org" :files (:defaults "contrib/lisp/.el") :build ("make" "oldorg" "EMACS=/usr/local/bin/emacs") :package "org-plus-contrib") nil)

f(compiled-function () #<bytecode 0x15613a30eac9>)()

straight--transaction-exec(use-package-(:type\ git\ :repo\ \"https://code\.orgmode\.org/bzg/org-mode\.git\"\ :local-repo\ \"org\"\ :files\ (:defaults\ \"contrib/lisp/.el\")\ :build\ (\"make\"\ \"oldorg\"\ \"EMACS=/usr/local/bin/emacs\")\ :package\ \"org-plus-contrib\")-nil-nil :now #f(compiled-function () #<bytecode 0x15613a30eac9>)) straight-use-package(org-plus-contrib) eval-buffer(#<buffer load-584805> nil "/home/rgr/.emacs.d/config/init.el" nil t) ; Reading at buffer position 460 load-with-code-conversion("/home/rgr/.emacs.d/config/init.el" "/home/rgr/.emacs.d/config/init.el" nil nil) load("/home/rgr/.emacs.d/config/init.el") eval-buffer(#<buffer load*> nil "/home/rgr/.emacs.d/init.el" nil t) ; Reading at buffer position 128 load-with-code-conversion("/home/rgr/.emacs.d/init.el" "/home/rgr/.emacs.d/init.el" t t) load("/home/rgr/.emacs.d/init" noerror nomessage) startup--load-user-init-file(#f(compiled-function () #<bytecode 0x15613a326285>) #f(compiled-function () #<bytecode 0x15613a326299>) t) command-line() normal-top-level()

My org form:

          (use-package org
            :straight org-plus-contrib
            :custom
            (org-directory "~/.emacs.d/orgfiles")
            (org-agenda-files (list org-directory (concat org-directory "/projects" )))
            (org-refile-targets `((,(directory-files-recursively org-directory "^[[:alnum:]].*\\.\\(org\\|gpg\\)\\'") :maxlevel . 9)))
            (org-outline-path-complete-in-steps nil)
            (org-refile-use-outline-path 'file)
            (org-refile-allow-creating-parent-nodes 'confirm)
            :config
            (require 'org-protocol)
            (require 'org-inlinetask)
            (defun rgr/org-agenda (&optional arg)
              (interactive "P")
              (let ((org-agenda-tag-filter-preset '("-trash")))
                (org-agenda arg "a")))
            :bind
            ("C-c a" . org-agenda)
            ("C-c A" . rgr/org-agenda)
            ("C-c c" . org-capture)
            ("C-c l" . org-store-link)
            ("C-c C-l" . org-insert-link)
            ("C-c C-s" . org-schedule)
            ("C-c t" . org-todo))

Did adding the :branch to the boxquote recipe fix the other issue for you?

Yes,partly. But there was another issue. It was the only one not working if my gpg agent was failing. Frustrating. It only started happening recently so I'll file this away as a debian startup issue for now.

progfolio commented 3 years ago

@rileyrg I cloned your emacs.d and played around to see what I could find. I'm fairly certain the issue is in the init-load procedure in config/init.el. You're calling org-babel-load-file which I believe is where the mix up is. I opened a pull with a fix that works locally for me. See if that fixes the rebuilding issue for you.

rileyrg commented 3 years ago

I'll try that this morning. Just an "fyi" this rebuilding business is quite new with "nothing" changed on my side. But now I think about it, the "fix", which was to hand eval the org form, ALSO prompted me to reload loaddefs. After that subsequent starts didnt rebuild.

postedit. I've never merged a pull request before so this might take some time. I need to howto it.

On Thu, 12 Nov 2020 at 08:10, Nicholas Vollmer notifications@github.com wrote:

@rileyrg https://github.com/rileyrg I cloned your emacs.d and played around to see what I could find. I'm fairly certain the issue is in the init-load config/init.el. You're calling org-babel-load-file which I believe is where the mix up is. I opened a pull with a fix that works locally for me. See if that fixes the rebuilding issue for you.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/raxod502/straight.el/issues/624#issuecomment-725886581, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACYTCZGZH47EN3MNZ3BKYDSPOC7LANCNFSM4TGXVASQ .

rileyrg commented 3 years ago

@rileyrg I cloned your emacs.d and played around to see what I could find. I'm fairly certain the issue is in the init-load procedure in config/init.el. You're calling org-babel-load-file which I believe is where the mix up is. I opened a pull with a fix that works locally for me. See if that fixes the rebuilding issue for you.

Sorry to report , it's still rebuilding. At first, I thought it was because I'd forgotten to recreate my init.elc. I pulled it into my "master" btw so you can see it there as I thought you'd like a quick turn around as opposed to me figuring out how to test it in a different branch (something I need to learn to do).

Hand evalling fixes it on subsequent restarts.

Screenshot from 2020-11-12 08-46-04 Screenshot from 2020-11-12 08-47-10

progfolio commented 3 years ago

There are a lot of layers of indirection in your .emacs.d set up. Would you mind walking me through how it is supposed to work? You mention an init.elc, but you're only ever loading init.el as far as I can tell.

https://github.com/rileyrg/Emacs-Customisations/blob/bb9bf8896a5888ee6aae60fb18c1a590b2fdda0b/init.el#L3

You've hardcoded the ".el" extension, perhaps you meant (load (expand-file-name "config/init" user-emacs-directory)) in which case load would prefer the .elc over the .el if they both exist.

rileyrg commented 3 years ago

The init.elc I mentioned was the one in the config/ directory. Funnily enough, I just renamed it emacs-config-init.el to avoid confusion with the top-level .emacs.d/init.el which emacs "auto finds". The top-level just loads the "real" init in config/ since time and time again emacs wrote over the symlink in .emacs.d directory when I merely symlinked config/init to .emacs.d/init. I have no doubt it can be simplified or cleaned up.

"You've hardcoded the ".el" extension, perhaps you meant (load (expand-file-name "config/init" user-emacs-directory)) in which case load would prefer the .elc over the .el if they both exist."

Thanks, I'll check/fix that.

On Thu, 12 Nov 2020 at 09:20, Nicholas Vollmer notifications@github.com wrote:

There are a lot of layers of indirection in your .emacs.d set up. Would you mind walking me through how it is supposed to work? You mention an init.elc, but you're only ever loading init.el as far as I can tell.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/raxod502/straight.el/issues/624#issuecomment-725919096, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACYTCZ2SXT6WYMKTHTQVJ3SPOLDTANCNFSM4TGXVASQ .

progfolio commented 3 years ago

I would argue that byte-compiling that file won't provide any meaningful performance gains either.

progfolio commented 3 years ago

I'm into the wee-hours of Eastern Time, so I've got to get to bed, but I'm 99% certain this is a configuration issue now. I'd be glad to help you sort it out and look over your emacs configuration if you'd like. I think simplifying a bit would make things easier to reason about and likely see a faster init time. Let me know and we can move the discussion to a separate issue on your repo.

rileyrg commented 3 years ago

I agree re byte-compiling and it's why I had the .emacs.d/init.el load the .emacs.d/config/init.el specifically now I think about it. Nice of you to offer. Let me run through it and tidy it if possible first as I don't want to burden you. But I'm not the only one with this rebuild issue I think.

re walkthrough, there's only one "redirection" and thats that .emacs.d/init.el exists to load the (now renamed) .emacs.d/config/init.el (emacs-config-init.el with my next push). Why? Because sometimes with a broken config emacs would wipe the .emacs.d/init.el symlink to .emacs.d/config/init.el file. Maybe it doesnt do it anymore. History etc.

jdek commented 3 years ago

I've been having the same issue, I used the following configuration in the past which worked, see https://github.com/raxod502/straight.el/issues/352#issuecomment-460069774.

(use-package org
  :straight (org-plus-contrib
             :repo "https://code.orgmode.org/bzg/org-mode.git"
             :local-repo "org"
             :files (:defaults "contrib/lisp/*.el")
             :includes (org)))

Switching to configuration which doesn't specify the repository url fixes it for me.

(use-package org :straight (org-plus-contrib :includes (org)))

How should org-mode be included with a pinned URL now?

progfolio commented 3 years ago

@jdek: The comment you've linked to shows a proposed :includes keyword, but that has not been implemented.

Please run the test case in this comment twice in a buffer with lexical binding enabled: https://github.com/raxod502/straight.el/issues/624#issuecomment-720153249

jdek commented 3 years ago

@progfolio Ah, I didn't realise it was a no-op. I think my case may have been related to being on an older Emacs (26.1 I think). I've just updated to 27.1 and the following seems to work:

(use-package org
  :straight (org-plus-contrib
             :repo "https://code.orgmode.org/bzg/org-mode.git"
             :local-repo "org"
             :files (:defaults "contrib/lisp/*.el"))

Thanks for the help.

Test Case ```emacs-lisp (straight-bug-report :user-dir "org-rebuild-test.straight" :preserve t :post-bootstrap (straight-use-package 'org)) ```
Output ```emacs-lisp Bootstrapping straight.el... Bootstrapping straight.el...done Rebuilding all packages due to build cache schema change Looking for gnu-elpa-mirror recipe → Cloning melpa... Looking for gnu-elpa-mirror recipe → Cloning melpa...done Looking for emacsmirror-mirror recipe → Cloning gnu-elpa-mirror... Looking for emacsmirror-mirror recipe → Cloning gnu-elpa-mirror...done Looking for straight recipe → Cloning emacsmirror-mirror... Looking for straight recipe → Cloning emacsmirror-mirror...done Building straight... Building straight...done Test run with version: prerelease (HEAD -> develop, origin/develop) 7b5a930 %cs Cloning org... Cloning org...done Building org... Building org...done ```
Test Case ```emacs-lisp (straight-bug-report :user-dir "org-rebuild-test.straight" :preserve t :post-bootstrap (straight-use-package 'org)) ```
Output ```emacs-lisp Test run with version: prerelease (HEAD -> develop, origin/develop) 7b5a930 %cs ```
progfolio commented 3 years ago

The output from the test runs looks good: no rebuild on the second run. Glad you got it working. Just out of curiosity, can I see the value of M-x straight-get-recipe org-plus-contrib?

jdek commented 3 years ago

Output of M-x straight-get-recipe org-plus-contrib

`(org-plus-contrib
 :type git
 :repo "https://code.orgmode.org/bzg/org-mode.git"
 :local-repo "org"
 :files (:defaults "contrib/lisp/*.el")
 :build ,(let ((make (if (eq system-type 'berkeley-unix) "gmake" "make"))
               (emacs (concat "EMACS=" invocation-directory invocation-name)))
           `(,make "oldorg" ,emacs)))