search

jwiegley / use-package

A use-package declaration for simplifying your .emacs
https://jwiegley.github.io/use-package
GNU General Public License v3.0
4.42k stars 260 forks source link

(Semi-)Manually load all deferred packages? #894

Open Atemu opened 3 years ago

Atemu commented 3 years ago

Deferring package loading is awesome for startup time but in some cases I don't care about startup time and/or want everything to be ready to use.

Use case 1:

Emacs started in daemon mode via systemd service; the meatbag typing in his secure password in takes longer than Emacs startup ever could.

Use case 1.5:

Using a deferredly loaded package for the first time tends to make Emacs hang for a bit and doesn't feel great. Especially annoying when it's been minutes/hours/days/weeks since emacs was started but the package still isn't hot.

Use case 2:

Native compile all packages AOT instead of JIT. Again, lagg and slowdown when loading a package by using it for the first time. This only happens after an emacsGcc update but is even worse.

It'd be great if there was:

I have tried this but it did not work for me (didn't actually load any additional packages, not sure if that's a bug?) and setting :defer f for all 270 packages my current spacemacs config draws in is not feasible.

conao3 commented 3 years ago

use-package defer package load just not generate require expression (pass this responsibility to Emacs native autoload feature), this is hard to solve your mentioned issue.

Atemu commented 3 years ago

I couldn't quite understand the first part of your sentence or its connection to the part in the parens. Could you reword it or write it in Japanese perhaps?

There's actually an update to this issue: Use case 1.5 can be solved with idle-loading.
This is achieved by setting :defer to the idle seconds after which the deferred package should be forcibly loaded (instead of just t).

https://github.com/jwiegley/use-package/blob/e6b4bf8458f94227ed64df6e8e873d147e42916f/use-package-core.el#L1582-L1586

conao3 commented 3 years ago

use-packageはパッケージのロードを遅らせるのを単にreuqire文を生成しないことで達成しています。(これはEmacsの autoload 機能にその責任 (defer loadする責任) を押し付ける) なので、この問題は解決するのは難しい。ということです。

:defer [N] で解決できるなら良かったです。

For English speakers, this comment has no addiional informaiton

Atemu commented 3 years ago

use-packageはパッケージのロードを遅らせるのを単にreuqire文を生成しないことで達成しています。

So use-package's deferred loading works by simply not requireing the packages after adding them to the list of packages?
That way they're in the list of available packages and whenever something explicitly requires a package, Emacs' autoload then takes care of loading it completely?

Would it be possible to record a list of the deferred packages as they are deferred perhaps? A user could then use that list to simply do an explicit require on all of those deferred packages.

I sort of got something similar working by mapping a partially applied require over the list of installed packages but that had issues with loading packages provided by the system and it'd generally be nicer to be able to map over only the packages that were actually deferred.

With a such a list of deferred packages, you could simply build things like the deferred idle-loading yourself and wouldn't have to rely on the package loading declaration to be done to your preferences.

:defer [N] で解決できるなら良かったです。

:defer [N] does indeed achieve the wanted effect but unfortunately it requires all :defer t to be changed to e.g. :defer 10 which isn't really feasible when working with pre-made configs like Spacemacs where defer t has been hard-coded for at least a hundred packages internally...

conao3 commented 3 years ago

If you catch all :defered packages I think you need to re-define :defer handler.

modified   use-package-core.el
@@ -1322,8 +1322,14 @@ meaning:
        `((run-with-idle-timer ,arg nil #'require
                               ',(use-package-as-symbol name) nil t)))
      (if (or (not arg) (null body))
-         body
-       `((eval-after-load ',name ',(macroexp-progn body)))))))
+         `((when (boundp 'use-package--defered-packages)
+             (defvar use-package--defered-packages nil))
+           (add-to-list 'use-package--defered-packages ',(use-package-as-symbol name))
+           ,@(macroexp-progn body))
+       `((when (boundp 'use-package--defered-packages)
+           (defvar use-package--defered-packages nil))
+         (add-to-list 'use-package--defered-packages ',(use-package-as-symbol name))
+         (eval-after-load ',name ',(macroexp-progn body)))))))

this generates Sexp to add its package symbol to use-package--defered-packages. After initialized your Emacs, you can require these packages something like this.

(dolist (pkg use-package--defered-packages)
  (require pkg))

you can also register this Sexp as a timer function.

skangas commented 1 year ago

Use case 1:

Emacs started in daemon mode via systemd service; the meatbag typing in his secure password in takes longer than Emacs startup ever could.

This use case should be covered with :hooks, :mode and similar, I think.

Use case 1.5:

Using a deferredly loaded package for the first time tends to make Emacs hang for a bit and doesn't feel great. Especially annoying when it's been minutes/hours/days/weeks since emacs was started but the package still isn't hot.

Use case 2:

Native compile all packages AOT instead of JIT. Again, lagg and slowdown when loading a package by using it for the first time. This only happens after an emacsGcc update but is even worse.

I see your point here, but isn't that covered using something like :defer 900 or similar?

So I'm not sure what more would be needed here, or why. Do you have a concrete example where the above wouldn't work?

Atemu commented 1 year ago

This use case should be covered with :hooks, :mode and similar, I think.

I don't see how? I've got a set of packages that are deferred indefinitely by Spacemacs (which is good for regular startup) but, under specific circumstances, I don't want them to be deferred.

I see your point here, but isn't that covered using something like :defer 900 or similar?

1.5 would be covered by that but the problem is that Spacemacs declares 100s of Packages as :defer t and I don't think I can influence that.

Also, I want to be able to AOT native-compile all of my packages when Emacs gets updated, not just the subset that has been used since the update.

skangas commented 1 year ago

This use case should be covered with :hooks, :mode and similar, I think.

I don't see how? I've got a set of packages that are deferred indefinitely by Spacemacs (which is good for regular startup) but, under specific circumstances, I don't want them to be deferred.

This is just regular autoloading, so the packages should be loaded when they are needed. But specifics matter, of course.

Do you have an example where it doesn't work?

I see your point here, but isn't that covered using something like :defer 900 or similar?

1.5 would be covered by that but the problem is that Spacemacs declares 100s of Packages as :defer t and I don't think I can influence that.

So can't you just add, for the packages you want to use, something like (use-package foo :defer 900)? That should add the appropriate idle timers to load the packages, no matter what Spacemacs is doing.

Also, I want to be able to AOT native-compile all of my packages when Emacs gets updated, not just the subset that has been used since the update.

I don't see how use-package could take care of that, it sounds like a job for package.el or similar. For what it's worth, I would solve this using :ensure and package-native-compile. Then I would just rm -rf ~/.emacs.d/elpa and restart Emacs. That would reinstall and AOT native compile all packages. Does that solution work for you?

Atemu commented 1 year ago

This is just regular autoloading, so the packages should be loaded when they are needed. But specifics matter, of course.

Do you have an example where it doesn't work?

Auto-loading on-use works and I want it in some cases but, in other cases, I want the packages to be manually-loaded all at once without ever needing to auto-load anything.

My example is that I use a daemon because and, in the time I need to get ready to use Emacs, the Emacs daemon will have already had time to load most packages; ready for me to use without delays or hitches.

So can't you just add, for the packages you want to use, something like (use-package foo :defer 900)? That should add the appropriate idle timers to load the packages, no matter what Spacemacs is doing.

I want all packages to have this property. That set of packages isn't stable and I don't want to be duplicating declarations.

I also don't know the implications of declaring a package twice, especially w.r.t. order.

I don't see how use-package could take care of that, it sounds like a job for package.el or similar.

The problem is that use-package is the one deferring loads and (presumably, I don't know a lot about how package loading works) has a list of packages that it deferred rather than load immediately.

I would solve this using :ensure and package-native-compile.

I am not sure what you mean by that.

Then I would just rm -rf ~/.emacs.d/elpa and restart Emacs. That would reinstall and AOT native compile all packages. Does that solution work for you?

I don't want to re-download all of my packages' sources again every time there is an Emacs update. That's a poor workaround rather than a solution.

I want to be able to say "All the packages that were installed but loading was deferred, explicitly load them now".

skangas commented 1 year ago

My example is that I use a daemon because and, in the time I need to get ready to use Emacs, the Emacs daemon will have already had time to load most packages; ready for me to use without delays or hitches.

You should use something like (use-package some-daemon) or (use-package some-daemon :demand t) for this.

I want all packages to have this property. That set of packages isn't stable and I don't want to be duplicating declarations.

How about use-package-defaults?

I also don't know the implications of declaring a package twice, especially w.r.t. order.

It has no implications, really. If the order matters, you do have to specify that again, of course.

I don't see how use-package could take care of that, it sounds like a job for package.el or similar.

The problem is that use-package is the one deferring loads and (presumably, I don't know a lot about how package loading works) has a list of packages that it deferred rather than load immediately.

Sorry, I don't think I understand what you're saying here. (use-package has no such list, FWIW.)

I would solve this using :ensure and package-native-compile.

I am not sure what you mean by that.

I mean that you would first want to set both of them to t, for the below to work:

Then I would just rm -rf ~/.emacs.d/elpa and restart Emacs. That would reinstall and AOT native compile all packages. Does that solution work for you?

I don't want to re-download all of my packages' sources again every time there is an Emacs update. That's a poor workaround rather than a solution.

Sorry, I thought you were asking what to do if you want to AOT all your packages after updating Emacs. The recipe above will do that reliably. (BTW, I don't think AOT compilation of all packages is very useful.)

Are you simply asking how to update your packages? You use package.el for that.

Otherwise, I don't follow.

I want to be able to say "All the packages that were installed but loading was deferred, explicitly load them now".

I don't understand when that would ever be useful. Autoloading and deferred loading seem to already cover this adequately.

In any case, you seem to be are asking about several separate issues, which makes the discussion hard to follow. Is this issue a feature request? If so, is this last quoted part the feature you are requesting?

skangas commented 1 year ago

My example is that I use a daemon because and, in the time I need to get ready to use Emacs, the Emacs daemon will have already had time to load most packages; ready for me to use without delays or hitches.

You should use something like (use-package some-daemon) or (use-package some-daemon :demand t) for this.

Sorry, I think I misunderstood you here, please disregard this part.

hab25 commented 9 months ago

@Atemu

I want to be able to say "All the packages that were installed but loading was deferred, explicitly load them now".

I use the following workround:

(map-do (lambda (key _) (require key)) use-package-statistics)

I consider it a workaround because:

hab25 commented 9 months ago

@skangas

I don't understand when that would ever be useful. Autoloading and deferred loading seem to already cover this adequately.

It is useful when using emacs --batch to test the :config blocks in one's init file.