Sourcing Zgen unnecessarily

[mfarrugi]

It shaves a bit of time to avoid sourcing zgen when you don't need to. Since that's kind of the point of this project, my .zshrc includes something like this:

if ! source "$HOME/.zgen/init.zsh"; then
    source "${HOME}/dots/zgen/zgen.zsh"
    #  Register plugins
    zgen save
    zgen init
fi

Maybe it would make sense to split out most of the generation logic, and source a lighter file that includes only zgen saved and zgen reset?

If there's no reason this hasn't already been done, I would be happy to do it.

[m42e]

Hi @mfarrugi.

I tried it (not like here) but integrate it into zgen. You may have a look at https://github.com/m42e/zgen/tree/splitfiles. (It includes my other changes as well)

Or try this one, on top of the current master https://github.com/m42e/zgen/tree/splitfile

Bye, Matthias

[tarjoilija]

I went this way in the beginning because I did some testing in shell and it was easy to have all zgen commands ready for use.

I really like this idea and I'll see if there's a good way of implementing this without breaking backwards compatibility.

[mfarrugi]

@m42e That looks like about what I would expect. (Btw, branch splitfile appears to have the change only partially applied.)

@tarjoilija What kind of invocations do you need to be backwards compatible with besides the example .zshrc?

source "${HOME}/proj/zgen/zgen.zsh"
if ! zgen saved; then
    echo "Creating a zgen save"

Pardon the obvious, but for this case we only need to source the zgen saved and zgen reset functions from zgen.zsh -- where zgen saved would conditionally load the rest of the functions. Does that break any other expected usage pattern?

[m42e]

@mfarrugi Well, I was a bit too fast it seems.

@tarjoilija The proposal doesn't break anything. Is simply delays loading of zgen_full until at least one command (or an invalid one)

[m42e]

@mfarrugi Have you tried it? Are you satisfied?

[mfarrugi]

No, but it looks fine to me. There's plenty of ways to do it :)

[ELLIOTTCABLE]

I've been doing something similar:

# I shim `zgen` to be lazy-loadable.
zgen() {
   local zgen_dir="$SYSTEM_REPO/Source/zgen"
   if [[ -r "$zgen_dir/zgen.zsh" ]]; then
      ZGEN_RESET_ON_CHANGE=("${HOME}/.zshrc" "${HOME}/.profile" "${HOME}/.profile.local")

      source "$SYSTEM_REPO/Source/zgen/zgen.zsh" >/dev/null && \
         zgen "$@"
   fi
}

if [[ ! -r "$HOME/.zgen/init.zsh" ]]; then
   printf %s\\n "-- zgen is caching Zsh modules. Shell re-load may be necessary."

   # ...
fi

source "$HOME/.zgen/init.zsh"

That (along with similar lazy-loading tactics for a couple other things like nvm and chruby) has seriously sped up my launch-time.

Actually, currently, the slowest part of my launch is actually init.zsh invoking shasum. I wonder if we could asynchronify that, somehow? Invoke it shortly after launch, so the user can immediately invoking commands (after all, if you're ⌘N'ing just to pop off a single quick command, you don't really care if your loaded Zsh configuration is up-to-the-minute, do you?)

[mfarrugi]

I think ZGEN_RESET_ON_CHANGE kind of defeats the purpose of managing a static configuration.. so I will personally continue to keep that empty.

As for speeding it up, we could compute a simpler hash such as filesize or time changed. Running it asynchronously would obviously work as well, but may be a little overcomplicated.

[m42e]

Well, you are right, you can disable the automatic check and it is disabled by default. :) You mean it should run the check in background and if the check fails it should delete the init.zsh? Well, maybe file size or change date is specific enough.

[mfarrugi]

I also narrowed down what exactly was slow..

Duration	% of Startup	Source	Event
23.4 ms	22.67%	zgen.zsh:474	compinit -i -C -d "${ZGEN_DIR}/zcompdump"
22.7 ms	21.99%	plugin	~/.zgen/.../oh-my-zsh.sh
11.4 ms	11.06%	zgen.zsh:467	ZSH=$(-zgen-get-zsh)
10.1 ms	9.80%	zgen.zsh:{1=>466}	Sourcing the file.
7.9 ms	7.63%	plugin	~/.zgen/.../zsh-history-substring-search.zsh

(Some of my other plugins for context)

Compinit obviously has to happen somewhere, but assigning ZSH can happen statically and sourcing the zgen.zsh file can be avoided :)

[willhoag]

This is pretty handy for anyone that want's to lazy load zgen: https://github.com/creationix/nvm/issues/539#issuecomment-110643090

lazy_source () {
    eval "$1 () { [ -f $2 ] && source $2 && $1 \$@ }"
}

lazy_source zgen ${HOME}/.zgen/zgen.zsh
if ! source "$HOME/.zgen/init.zsh"; then
    ...
fi

[jedahan]

@mfarrugi how did you get that nice benchmark?

[mfarrugi]

Here's a gist for it.

It's not very pretty, but hey I was profiling a shell script :)

[jedahan]

Thanks!

[Tuurlijk]

This is what i currently use: https://github.com/Tuurlijk/dotfiles/blob/master/.zshrc#L15

# Load zgen only if a user types a zgen command
zgen () {
    if [[ ! -s ${ZDOTDIR:-${HOME}}/.zgen/zgen.zsh ]]; then
        git clone --recursive https://github.com/tarjoilija/zgen.git ${ZDOTDIR:-${HOME}}/.zgen
    fi
    source ${ZDOTDIR:-${HOME}}/.zgen/zgen.zsh
    zgen "$@"
}

# Generate zgen init script if needed
if [[ ! -s ${ZDOTDIR:-${HOME}}/.zgen/init.zsh ]]; then
    zgen load zsh-users/zsh-autosuggestions
    zgen load bric3/nice-exit-code
    zgen load zdharma/fast-syntax-highlighting
    zgen load zsh-users/zsh-history-substring-search
    zgen oh-my-zsh plugins/shrink-path
    #zgen oh-my-zsh plugins/tmux
    zgen save
    zcompile ${ZDOTDIR:-${HOME}}/.zgen/init.zsh
fi

Then I just leave the file there in the ~/.zgen dir.

At the end I concatenate the .zgen/init.zsh and all the other files I have in ~/.config/zsh/ into a single cached file:

# Load settings
if [[ ! -s ${ZDOTDIR:-${HOME}}/.config/zsh/cache/settings.zsh ]]; then
    source ${ZDOTDIR:-${HOME}}/.config/zsh/functions.zsh
    recreateCachedSettingsFile
fi
source ${ZDOTDIR:-${HOME}}/.config/zsh/cache/settings.zsh

You can find the cache function here: https://github.com/Tuurlijk/dotfiles/blob/master/.config/zsh/functions.zsh#L56

I'm not yet happy with the glob. I would like to have a blacklist there instead of the current whitelist. But it works nicely.

Timings on a server: repeat 20 {time zsh -i -c exit}

zsh -i -c exit  0.04s user 0.00s system 85% cpu 0.043 total
zsh -i -c exit  0.04s user 0.00s system 87% cpu 0.042 total
zsh -i -c exit  0.03s user 0.01s system 91% cpu 0.044 total