search

radian-software / straight.el

🍀 Next-generation, purely functional package manager for the Emacs hacker.
MIT License
2.71k stars 150 forks source link

straight is rebuilding org on every single launch of emacs #624

Closed aspiers closed 3 years ago

aspiers commented 3 years ago

What's wrong

I am now seeing a Building org... message from straight.el every single time I launch emacs. Presumably this is related to the recent addition of support for :build steps. Since the build steps do things like compiling the whole manual, this takes something like one or two minutes and therefore drastically increases the waiting time before emacs is ready for use.

Directions to reproduce

I've added some debug to the bit of straight-use-package which calculates modified and it is showing that (straight--package-might-be-modified-p recipe no-build) returns 98, so presumably it's something in there which is going wrong.

But if you can't reproduce this and it's looking like this might be related to some peculiarity of my config, given that I have a decent amount of elisp hacking experience, rather than trying to create a minimal test case I guess it might be quicker if you can confirm I'm on the right track and I'll keep digging in straight--package-might-be-modified-p.

Version information

progfolio commented 3 years ago

Thank you

jdek commented 3 years ago

I have this issue again. A rm -rf ~/.emacs.d/straight/{build,build-cache.el,repos} sometimes fixed it. There is no visible issues again using the test-cases (apart from the first test-case failing for some reason). On macOS, exec-path-from-shell is required to correct the PATH to include the package manager's makeinfo command as the built-in version is too old to build org-mode.

Minimal Config ```emacs-lisp (defvar bootstrap-version) (let ((bootstrap-file (expand-file-name "straight/repos/straight.el/bootstrap.el" user-emacs-directory)) (straight-repository-branch "develop") (bootstrap-version 5)) (unless (file-exists-p bootstrap-file) (with-current-buffer (url-retrieve-synchronously "https://raw.githubusercontent.com/raxod502/straight.el/develop/install.el" 'silent 'inhibit-cookies) (goto-char (point-max)) (eval-print-last-sexp))) (load bootstrap-file nil 'nomessage)) (straight-use-package '(use-package :type git :host github :repo "jwiegley/use-package")) (use-package exec-path-from-shell :if (memq window-system '(mac ns)) :straight (:host github :repo "purcell/exec-path-from-shell") :init (exec-path-from-shell-initialize) :config ;; Seems broken after 27.1 (setq default-directory "~/") (setq command-line-default-directory "~/")) (use-package org :straight (org-plus-contrib :repo "https://code.orgmode.org/bzg/org-mode.git" :local-repo "org" :files (:defaults "contrib/lisp/*.el"))) ```

This doesn't occur with the minimal configuration above. If I then switch to my config from the minimal config in-place without deleting anything then org-mode is not rebuilt. I assume this means something in my config is touching org-mode's files timestamps but only sometimes? (also only org-mode nothing else is ever rebuilt)

Test cases

Test Case 1 ```emacs-lisp (straight-bug-report :user-dir "org-rebuild-test.straight" :preserve t :post-bootstrap (straight-use-package 'org)) ```
Output ```emacs-lisp Bootstrapping straight.el...done Bootstrapping straight.el... Bootstrapping straight.el...done Rebuilding all packages due to build cache schema change Looking for gnu-elpa-mirror recipe → Cloning melpa... Looking for emacsmirror-mirror recipe → Cloning gnu-elpa-mirror... Looking for emacsmirror-mirror recipe → Cloning gnu-elpa-mirror...done Looking for straight recipe → Cloning emacsmirror-mirror... Looking for straight recipe → Cloning emacsmirror-mirror...done Building straight... Building straight...done Test run with version: prerelease (HEAD -> develop, origin/develop) 7b5a930 %cs Cloning org... Looking for gnu-elpa-mirror recipe → Cloning melpa...done Building straight... Building straight...done Test run with version: prerelease (HEAD -> develop, origin/develop) 7b5a930 %cs Building org... Failed to run "make": make: *** No rule to make target `oldorg'. Stop. Warning (straight): Failed to run "make"; see buffer *straight-process* Building org...done ```
Test Case 2 ```emacs-lisp (straight-bug-report :user-dir "org-rebuild-test.straight" :preserve t :post-bootstrap (straight-use-package 'org)) ```
Output ```emacs-lisp Test run with version: prerelease (HEAD -> develop, origin/develop) 7b5a930 %cs Building org... Building org...done ```
Test Case 3 ```emacs-lisp (straight-bug-report :user-dir "org-rebuild-test.straight" :preserve t :post-bootstrap (straight-use-package 'org)) ```
Output ```emacs-lisp Test run with version: prerelease (HEAD -> develop, origin/develop) 7b5a930 %cs ```
progfolio commented 3 years ago

@jdek: Sorry to hear that. It most likely is a configuration error. If you would like to link me to your config, I'd be happy to take a look at it.

progfolio commented 3 years ago

Appreciate everyone's patience. This is a thorny one. Here's what I'm thinking is at the root of all this:

I'll implement #76 first and keep everyone posted. Thanks again for your patience and the information you've provided.

kevinjfoley commented 3 years ago

Looks like you have a plan of action on this @progfolio, sharing an additional test case below in case it's helpful.

Also it seems like some people have resolved when testing with straight-bug-report only to still experience the issue with their config. I found that if I changed the test case code without using a fresh :user-dir it seemed to trick straight-bug-report. But as soon as I ran the same test case twice in a fresh :user-dir the problem would present it self again.

Not sure what the cause is but thought it might be an explanation why the problem seems to go away when testing.

Test Case ```emacs-lisp (straight-bug-report :user-dir "org-contrib-rebuild-test.straight" :preserve t :post-bootstrap (straight-use-package 'use-package) (use-package org :straight (org-plus-contrib :repo "https://code.orgmode.org/bzg/org-mode.git" :local-repo "org" :files (:defaults "contrib/lisp/*.el"))) (use-package org-super-agenda :straight t :after org)) ```
First Run Output ```emacs-lisp Bootstrapping straight.el... Bootstrapping straight.el...done Rebuilding all packages due to build cache schema change Looking for gnu-elpa-mirror recipe → Cloning melpa... Looking for gnu-elpa-mirror recipe → Cloning melpa...done Looking for emacsmirror-mirror recipe → Cloning gnu-elpa-mirror... Looking for emacsmirror-mirror recipe → Cloning gnu-elpa-mirror...done Looking for emacsmirror-mirror recipe → Cloning el-get... Looking for emacsmirror-mirror recipe → Cloning el-get...done Looking for straight recipe → Cloning emacsmirror-mirror... Looking for straight recipe → Cloning emacsmirror-mirror...done Building straight... Building straight...done Test run with version: prerelease (HEAD -> develop, origin/develop) 949a0a7 2020-12-13 Cloning use-package... Cloning use-package...done Building use-package... Building use-package → Building bind-key... Building use-package → Building bind-key...done Building use-package... Building use-package...done Cloning org (for org-plus-contrib)... Cloning org (for org-plus-contrib)...done Building org-plus-contrib... Building org-plus-contrib...done Cloning org-super-agenda... Cloning org-super-agenda...done Building org-super-agenda... Building org-super-agenda → Cloning s.el... Building org-super-agenda → Cloning s.el...done Building org-super-agenda → Building s... Building org-super-agenda → Building s...done Building org-super-agenda → Cloning dash.el... Building org-super-agenda → Cloning dash.el...done Building org-super-agenda → Building dash... Building org-super-agenda → Building dash...done Building org-super-agenda → Building org... Building org-super-agenda → Building org...done Building org-super-agenda → Cloning ht.el... Building org-super-agenda → Cloning ht.el...done Building org-super-agenda → Building ht... Building org-super-agenda → Building ht...done Building org-super-agenda → Cloning ts.el... Building org-super-agenda → Cloning ts.el...done Building org-super-agenda → Building ts... Building org-super-agenda → Building ts...done Building org-super-agenda... Building org-super-agenda...done ```
Second Run Output ```emacs-lisp Test run with version: prerelease (HEAD -> develop, origin/develop) 949a0a7 2020-12-13 Building org-plus-contrib... Building org-plus-contrib...done Building org... Building org...done ```
progfolio commented 3 years ago

The latest develop branch is byte compiling in a separate process and has the altered Org mode recipes. Hoping that takes care of this. Appreciate any testing/feedback.

aaronjensen commented 3 years ago

This appears to work for me. I had to update my custom org-plus-contrib recipe to use the new built-in one, which appears to work well. Thank you!

rileyrg commented 3 years ago

I'll gladly test later. What's the recommended use-package/straight-use form for this now?

On Sat, 2 Jan 2021 at 07:54, Aaron Jensen notifications@github.com wrote:

This appears to work for me. I had to update my custom org-plus-contrib recipe to use the new built-in one, which appears to work well. Thank you!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/raxod502/straight.el/issues/624#issuecomment-753440788, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACYTC2OEXBNFVV6OWKHP2TSX27MBANCNFSM4TGXVASQ .

aspiers commented 3 years ago

I just switched to (use-package org-plus-contrib) and other packages to :after org-plus-contrib and it seems to work fine.

rileyrg commented 3 years ago

but org-plus-contrib takes care of loading org? What a minefield. I've kind of given up trying to understand it;)

On Sat, 2 Jan 2021 at 12:14, Adam Spiers notifications@github.com wrote:

I just switched to (use-package org-plus-contrib) and other packages to :after org-plus-contrib and it seems to work fine.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/raxod502/straight.el/issues/624#issuecomment-753461156, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACYTCZRXSPEUNRMBGZPKODSX353BANCNFSM4TGXVASQ .

aspiers commented 3 years ago

If you look at the latest source, you'll see that straight.el translates both org and org-plus-contrib into recipes which start with (org ...) and both have "lisp/*.el" in their :files property, but only org-plus-contrib also has "contrib/lisp/*.el" in its :files property.

rileyrg commented 3 years ago

I've no idea what that means or the consequences. I'll just use-package org-plus-contrib and see how it goes. Many thanks.

On Sat, 2 Jan 2021, 14:22 Adam Spiers, notifications@github.com wrote:

If you look at the latest source, you'll see that straight.el translates both org and org-plus-contrib into recipes which start with (org ...) and both have "lisp/.el" in their :files property, but only org-plus-contrib also has "contrib/lisp/.el" in its :files property.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/raxod502/straight.el/issues/624#issuecomment-753473136, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACYTC7O33ZK5KBVUMLK2P3SX4MZVANCNFSM4TGXVASQ .

progfolio commented 3 years ago

Thank you everyone for you patience and testing efforts. Glad to see things are working. I'll consider this bug closed for now. If the issue crops up again for anyone, feel free to leave a comment and we can reopen it.

@reilyrg: I understand the confusion around Org vs "Org plus contrib". In Org 9.5, the plan (as I understand it) is to break out the contrib packages into their own separate repository. This should simplify things and hopefully clear up some of the confusion. If you're interested in the details:

https://orgmode.org/list/87wnzfy60h.fsf@bzg.fr/

raxod502 commented 3 years ago

Thank goodness. org-plus-contrib is quite frankly bizarre. Unfortunately, in the absence of some upstream packaging changes, I do not think it is possible to smooth over the unintuitive behavior in straight.el (really, org-plus-contrib should be a virtual package consisting of "org" and "org-contrib" separately, but there's no concept of virtual packages or anything like that).

@progfolio One thing I notice with the new recipes is that it is now possible for (straight--convert-recipe 'org-plus-contrib) to return a recipe whose package is not named org-plus-contrib -- I believe such a situation never happened before. I don't know if that would cause any problems, and clearly your implementation is working well enough so far, but I wanted to call your attention to this in case something does crop up later.

aaronjensen commented 3 years ago

I don't know if it's something particular to my setup, but I still frequently have to rebuild org.

aaronjensen commented 3 years ago

I tracked this down. Because I launch Emacs on a mac in different ways, invocation-directory and invocation-name can vary. When they vary, org rebuilds because of:

https://github.com/raxod502/straight.el/blob/e98ea33388196bfe30c7a69e2abd5a93a8a701f8/straight.el#L2820

Instead, I hardcoded the recipe with EMACS=emacs which is sufficient for me and it no longer rebuilds unnecessarily

(straight-use-package
 `(org
   :type git
   :repo "https://code.orgmode.org/bzg/org-mode.git"
   :depth full
   :build (:not autoloads)
   :local-repo "org"
   :pre-build ,(list
                (concat (when (eq system-type 'berkeley-unix) "g")
                        "make")
                "autoloads"
                "EMACS=emacs")
   :files (:defaults "lisp/*.el"
                     "contrib/lisp/*.el")))
raxod502 commented 3 years ago

Maybe those variables should be evaluated at build time, rather than at recipe expansion time? I can't think of a good reason we'd want to rebuild Org just because a different path to Emacs is used. Perhaps if it's a different Emacs version, but in that case the build cache will be invalidated anyway.

progfolio commented 3 years ago

Agreed. I'll look into it.