ZSH config file is really heavy

[Phyks]

Hi,

I tried to source $(pew shell_config) in my .zshrc to have a feedback on the venv I am using, in my prompt. However, it adds 2 extra seconds or so to spawn a new ZSH shell, which is a lot for such a basic feature.

Is it normal? Expected?

Thanks

EDIT: Plus it messes with my PS1, breaking new lines etc.

[berdario]

The PS1 problem was #69

That one has been fixed a while ago, but I didn't release a new version until yesterday.

I'm not sure about the delay, it's definitely not expected, but maybe it's related? Does it still happen with the latest version?

Thank you

[Phyks]

@berdario Indeed, PS1 problem seems to be #69 and is fixed.

Delay is still present however =( I can help you in debugging it, but I do not really know what could be useful to you.

Thanks

[Phyks]

Let me know if you have any test in mind that I could do to debug it further.

Here is a quick comparison on another laptop, same issue:

Without the source line:

% time zsh -c "source /home/lucas/.zshrc"
zsh -c "source /home/lucas/.zshrc"  0,05s user 0,01s system 103% cpu 0,060 total

With the source line:

% time zsh -c "source /home/lucas/.zshrc"
zsh -c "source /home/lucas/.zshrc"  0,38s user 0,11s system 99% cpu 0,494 total

[berdario]

Thanks

The difference seems to be only 0.43s, definitely not ideal, but not as bad as 2 extra seconds

I would try to pin down if the culript is the Python startup, compinit, or something else... remove the source $(pew shell_config) line from your .zshrc and let's try something like this:

dario@curie ~> time zsh -ic 'pew shell_config'
/home/dario/.local/venvs/pew/lib/python3.5/site-packages/pew/shell_config/init.fish
0.19user 0.02system 0:00.24elapsed 89%CPU (0avgtext+0avgdata 23144maxresident)k
0inputs+0outputs (0major+13131minor)pagefaults 0swaps

dario@curie ~> time zsh -ic 'compinit'
0.05user 0.02system 0:00.10elapsed 77%CPU (0avgtext+0avgdata 5732maxresident)k
0inputs+0outputs (0major+6831minor)pagefaults 0swaps

dario@curie ~> time zsh -ic 'source /home/dario/.local/venvs/pew/lib/python3.5/site-packages/pew/shell_config/complete.zsh; compinit'
_arguments:comparguments:312: can only be called from completion function
0.06user 0.02system 0:00.10elapsed 78%CPU (0avgtext+0avgdata 5780maxresident)k
0inputs+0outputs (0major+6863minor)pagefaults 0swaps

I just had to install zsh to try out these commands (I don't usually use zsh) and I'm getting this

 _arguments:comparguments:312: can only be called from completion function

error, so the completion script is not loaded successfully and the timings will obviously be skewed... anyhow, I suspect that the culprit might be compinit, if a lot of other completion functions have been loaded before

[Phyks]

Ok, here are the results.

% /usr/bin/time -p zsh -ic "source /tmp/.zshrc_with_source"
real 0.86
user 0.65
sys 0.21

% /usr/bin/time -p zsh -ic "source /tmp/.zshrc"            
real 0.37
user 0.28
sys 0.10

% /usr/bin/time -p zsh -ic "pew shell_config"
/usr/lib/python2.7/site-packages/pew/shell_config/init.zsh
real 0.25
user 0.18
sys 0.07

% /usr/bin/time -p zsh -ic "compinit"
real 0.14
user 0.09
sys 0.04

% /usr/bin/time -p zsh -ic "source /usr/lib/python2.7/site-packages/pew/shell_config/init.zsh; compinit"
real 0.40
user 0.30
sys 0.11

I used the standard GNUTime as I think it is a more standard way to measure time and it seems to return times which are much more in agreement with the things I observe.

Actually, even if the difference in start time with or without sourcing the pew shell config seems to be small, without the source line I have directly a shell when I open a terminal, whereas when I add the source line, there is a noticeable time between loading the terminal and having a working shell. I may be able to record a screencast if you want to see the difference by yourself.

[berdario]

Ok

The blame seems to be for 2/3 on the completion script/compinit and for 1/3 on the Python/Pew startup time itself

If the only reason you're sourcing the shell helper is to know which venv you're currently using, you can simply lift only the bare minimum needed code from /usr/lib/python2.7/site-packages/pew/shell_config/init.zsh:

Which should be

function virtualenv_prompt_info() {
    if [ -n "$VIRTUAL_ENV" ]; then
        local name=$(basename $VIRTUAL_ENV)
        echo "($name)"
    fi
}
PS1="$(virtualenv_prompt_info)$PS1"

I'm not sure if something else could/should be done... One day I'll try to have a look if there's a difference in the additional startup time added to the different shells, and if zsh if out of line here.

The main purpose of the shell_config is for autocompletion, and thus disabling it does not really make sense. I'm not the original author of complete.zsh and I'm not a zsh user, so I'm not sure if there's any obvious improvement that could be done there.

Adding the source $(pew shell_config) line is optional, and the first_run_setup will default to not add it. Maybe I could add a message warning about the potentially increased shell startup time? Or maybe if people see the source $(pew shell_config) as a bit of a black-box something could be added in the README to invite users to adapt it to their needs?

[Phyks]

On another machine:

19:51 phyks@Phyks-laptop ~ % time zsh -ic "source ~/.zshrc"
zsh -ic "source ~/.zshrc"  1,07s user 0,12s system 96% cpu 1,235 total

19:51 phyks@Phyks-laptop ~ % time zsh -ic "source ~/.zshrc"
zsh -ic "source ~/.zshrc"  0,23s user 0,04s system 95% cpu 0,277 total

(first one is with the source line)

After further tests, major source of overhead is the pew shell_config line which invokes a Python shell. Replacing source $(pew shell_config) by source PATH_TO_INIT_ZSH gets a lot better.

Finally, using compinit in init.zsh might be useless as users already run it in their .zshrc. I am not 100% sure about the reason it is here.

Anyway, putting the actual content you mentionned in my zshrc is already what I do and it works just fine.

[berdario]

Yeah, the path outputted by shell_config differs depending on how/where pew has been installed...

The idea is also that when pew is updated, the new version of any completion script will be thus used in the user's shell config

That's pretty much a case of:

All problems in computer science can be solved by another level of indirection ... except for the problem of too many layers of indirection

I guess I could offer an option:

• add source $(pew shell_config)
• add (hardcoded) source PATH_TO_INIT_ZSH
• [default] don't modify shell config file

Do you have any other idea?

[Phyks]

Apart from turning pew shell_config into a sh script, or something like this, I do not see many other solutions. Sorry…

[ashwinvis]

An old issue, but I totally agree with @Phyks.

As a solution, I found prezto's python module to be much more efficient. What's more, the virtualenv information is displayed on the right side of the terminal. Somehow it works with pew, although it only supports virtualenvwrapper and conda.

Here are some timings:

with source $(pew shell_config):

~ ❯❯❯ time zsh -ic "source ~/.zshrc" 
zsh -ic "source ~/.zshrc"  1.12s user 0.48s system 100% cpu 1.604 total

with prezto's python module

~ ❯❯❯ time zsh -ic "source ~/.zshrc"                      
zsh -ic "source ~/.zshrc"  0.11s user 0.11s system 99% cpu 0.219 total

[mjlbach]

@ashwinvis There are two ways to speed up a shell start using pew.

1. If you're using a framework like Oh My Zsh or Prezto that loads compinit, you can see a large speedup by deleting compinit from line 2 of PEW's init.zsh file. For example, mine is located at "/usr/local/lib/python3.6/site-packages/pew/shell_config/init.zsh" As long as pew is sourced after the framework (at the end of .zshrc in my case) this will result in a speedup of roughly 0.7 seconds without losing completion. Loading completion is one of the biggest slowdowns with shell startup.

2. If you use the python2 package for pew, shell start-up is about 0.2 seconds faster, presumably because python 2 still starts up faster than python 3 (even 3.7 by ~2 fold)