Slowness on first cd - Githubissues

clee231 commented 7 years ago

I've noticed that sexy-bash-prompt causes the terminal to wait for some time when visiting a git repository initially. After about ~20 seconds, control is given back to the user. Subsequent visits to that same directory throughout the day seem to be instantanteous. If you don't visit the directory for a prolonged time (say 12 hours), the slowness still occurs.

I've been trying to do some testing, but since cd isn't really a executable command, I can't simply chain two date commands in the same line.

Below is a simple test that I ran (see image): Imgur

I've noticed that if I copy a repository that I haven't visited in a while, the same slow behavior occurs when I cd into that copied directory. I'm thinking it has something to do with caching, but I'm not too sure.

rpdelaney commented 7 years ago

What's your OS and bash version?

I've seen the same behavior on cygwin, but never in unix. I have some theories about how that could be replicated though.

clee231 commented 7 years ago

This particular VM is running Red Hat Enterprise Linux Server release 5.11 (Tikanga). The bash version is GNU bash, version 3.2.25(1)-release (x86_64-redhat-linux-gnu).

This version of bash is the one provided by the yum package manager (EPEL).

$yum info bash
Loaded plugins: rhnplugin, security
Installed Packages
Name       : bash
Arch       : x86_64
Version    : 3.2
Release    : 33.el5_11.4
Size       : 5.0 M
Repo       : installed
Summary    : The GNU Bourne Again shell (bash) version 3.1.
URL        : http://www.gnu.org/software/bash
License    : GPLv2+
Description: The GNU Bourne Again shell (Bash) is a shell or command language
           : interpreter that is compatible with the Bourne shell (sh). Bash
           : incorporates useful features from the Korn shell (ksh) and the C shell
           : (csh). Most sh scripts can be run by bash without modification. This
           : package (bash) contains bash version 3.1, which improves POSIX
           : compliance over previous versions

rpdelaney commented 7 years ago

Thanks. sexy-bash-prompt runs a few different git commands on each invocation to gather information. In the past I've had performance issues with git rev-parse --git-dir (part of sexy_bash_prompt_get_git_progress()). Can you do some testing with timing that command? The time command is useful for that purpose, e.g.:

export PS1="$ "
cd git_dir
time git rev-parse --git-dir

twolfson commented 7 years ago

It's also possible the performance issues are coming from the VM's file system. I've had issues with content being synced in large folders (e.g. ones that contain node_modules) albeit that's a bit of a different problem. What is the VM's underlying file system?

clee231 commented 7 years ago

@rpdelaney - I don't think the rev-parse command is causing the slowness. I tried calling time on each sexy_bashprompt* function and here are the results:

$ cd portal2
$ time git rev-parse --git-dir
.git

real    0m0.012s
user    0m0.001s
sys 0m0.006s

$ time sexy_bash_prompt_get_git_branch
master

real    0m0.066s
user    0m0.000s
sys 0m0.009s
$ time sexy_bash_prompt_get_git_progress

real    0m0.002s
user    0m0.001s
sys 0m0.001s
$ time sexy_bash_prompt_is_branch1_behind_branch2 

real    0m0.010s
user    0m0.001s
sys 0m0.004s
$ time sexy_bash_prompt_branch_exists 

real    0m0.061s
user    0m0.002s
sys 0m0.003s
$ time sexy_bash_prompt_parse_git_ahead 
1

real    0m0.016s
user    0m0.004s
sys 0m0.011s
$ time sexy_bash_prompt_parse_git_behind 

real    0m0.008s
user    0m0.006s
sys 0m0.004s
$ time sexy_bash_prompt_parse_git_dirty

real    0m18.822s
user    0m1.041s
sys 0m0.498s
$ time sexy_bash_prompt_is_on_git

real    0m0.003s
user    0m0.000s
sys 0m0.003s
$ time sexy_bash_prompt_get_git_status
△

real    0m0.097s
user    0m0.035s
sys 0m0.062s
$ time sexy_bash_prompt_get_git_info
master△

real    0m0.106s
user    0m0.048s
sys 0m0.063s

We see that sexy_bash_prompt_parse_git_dirty() is the function that takes the longest to run. Looking at this function, I see that git status --porcelainis being run. I verified that the first run of this command is significantly slower than subsequent calls. I guess this just might be a problem with git that I have do deal with.

@twolfson - That's totally possible as well. I don't personally manage this VM, but the mount command reports that this is an ext3 file system.

twolfson commented 7 years ago

Ah, k. ext3 is fine -- if it was something like nfs then that might explain the behavior although I lack experience working with network mounted drives =/ My thought was:

OS hadn't cached files locally yet
On first directory load, it's loading all files to run git status on them
On subsequent calls, it has files cached locally

Since there seems to be no obvious cause, let's keep on digging. Here are some more questions:

What happens when you run git status without --porcelain?
What version of git are you using? (git --version)
How many files are in the repository? (find . -type f | grep --invert-match ./.git | wc -l)
- Attribution: http://askubuntu.com/a/34102
How large is the git repository? (du -h --max-depth 1 .git)

clee231 commented 7 years ago

With --porcelain:

$ time git status --porcelain

real    0m23.486s
user    0m1.334s
sys 0m0.749s

Without --porcelain:

$ time git status
# On branch master
nothing to commit (working directory clean)

real    0m26.174s
user    0m1.227s
sys 0m0.768s

The git version is 1.8.2.1. There are currently 12422 files in the repository.

The size of the .git repository is as follows:

$ du -h --max-depth 1 .git
48K .git/hooks
90M .git/objects
72K .git/refs
132K    .git/logs
8.0K    .git/info
4.0K    .git/branches
91M .git

rpdelaney commented 7 years ago

That's a pretty old git version. I have no idea if a later version would improve the performance of git status though. Regardless, this is pretty clearly not a bug with sexy-bash-prompt.

twolfson commented 7 years ago

Ah, damn. I forgot that the files count could include external files. Can you try git ls-files | wc -l for counting those?

Also, maybe try the following to reduce the suspects:

Determine if disk reads are slow by combining ls -la with the git ls-files results
- time git ls-files > tmp.txt
- time ls -la $(cat tmp.txt)
Try out a newer version of git as @rpdelaney suggests. If you can't install a system level one (you mentioned the VM not being yours), you should be able to use a locally compiled binary
- https://git-scm.com/book/en/v1/Getting-Started-Installing-Git

clee231 commented 7 years ago

It seems that Red Hat Enterprise Linux 5(RHEL5) Extra Packages for Enterprise Linux(EPEL) only has a package of git up to 1.8.2.3. I'll try seeing if one of the sysadmins can update to this version. If that doesn't help, then compiling git from source would probably be the next best option... (although it's probably not supported by RedHat support.)

clee231 commented 7 years ago

git ls-files | wc -l returns: 6412 files

$ time git ls-files > tmp.txt

real    0m0.014s
user    0m0.003s
sys     0m0.006s

$ time ls -la $(cat tmp.txt)
real    0m0.605s
user    0m0.212s
sys     0m0.090s

I don't think the lag is attributed to slow disk reads. Those timings seem pretty reasonable. It is most definitely an issue with Git as @rpdelaney noted. Hopefully a new version will fix this issue.

rpdelaney commented 7 years ago

Another thing worth trying is doing some git housekeeping: git gc

twolfson / sexy-bash-prompt

Slowness on first cd #71