twpayne / chezmoi

Manage your dotfiles across multiple diverse machines, securely.
https://www.chezmoi.io/
MIT License
12.63k stars 475 forks source link

Unnecesary slow operations #2354

Closed Lockszmith-GH closed 1 year ago

Lockszmith-GH commented 1 year ago

Describe the bug

I'm on a cruise ship in vacation, had to pull out my laptop to perform some quick operation. My terminal shell-initialization script was not initializing, and I discovered that it was stuck on a call to chezmoi source-path which I store in an environment variable.

Using the --verbose and --debug flags, it shows that chezmoi is performing some very slow operations even when it should not.

Problem (1) - the somewhat severe: The operation lingers for ~11 seconds with a ReadFile of a cached binary file. (one-before last line in output below)

Problem (2) - the VERY severe one In my .chezmoiexternal.toml I have a reference to some external resources, and those are queried which causes the operation to hang - because of my slow connection.

To reproduce

Connect to a VERY SLOW internet connection (I believe completely offline will just fail the calls, and will not reproduce it) and run chezmoi source-path

Expected behavior

A quick operation, returning the source-path, never reaching out to online resources, or at least a way to flag to chezmoi that we're offline.

Output of command with the --verbose flag

$ chezmoi source-path --debug --verbose --dry-run                                                                                                                                                           
2022-09-13T14:02:03-04:00 INF persistentPreRunRootE args=["chezmoi","source-path","--debug","--verbose","--dry-run"] goVersion=go1.18.2 version={"builtBy":"goreleaser","commit":"565cbbe117746aa6bfec5f2cee
20ae4cbbb5e645","date":"2022-05-30T10:24:34Z","version":"2.17.1"}                                                                                                                                           
2022-09-13T14:02:03-04:00 INF Stat component=system name=<$HOME>/.local/share/chezmoi/.git                                                                                                                
2022-09-13T14:02:03-04:00 ERR ReadFile error="open <$HOME>/.local/share/chezmoi/.chezmoiversion: no such file or directory" component=system data= name=<$HOME>/.local/share/chezmoi/.chezmoiversion siz
e=0                                                                                                                                                                                                         
2022-09-13T14:02:03-04:00 INF ReadFile component=system data="_home\n" name=<$HOME>/.local/share/chezmoi/.chezmoiroot size=6                                                                              
2022-09-13T14:02:03-04:00 INF Stat component=system name=<$HOME>/.local/share/chezmoi/_home                                                                                                               
2022-09-13T14:02:03-04:00 INF Stat component=system name=<$HOME>/.local/share/chezmoi/_home                                                                                                               
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home                                                                                                            
2022-09-13T14:02:03-04:00 INF ReadFile component=system data="{{ if eq .chezmoi.hostname \"laptop-hostname\" }}\n[\".local/share/git-on..." name=<$HOME>/.local/share/chezmoi/_home/.chezmoiexternal.toml size=
3438                                                                                                                                                                                                        
2022-09-13T14:02:03-04:00 INF ReadFile component=system data="NAME=\"Pop!_OS\"\nVERSION=\"22.04 LTS\"\nID=pop\nID_LIKE=\"ubuntu debian..." name=/etc/os-release size=374                                    
2022-09-13T14:02:03-04:00 INF ReadFile component=system data=".local/share/fonts/**/.uuid\n.ssh/*\n~.ssh/authorized_keys.templat..." name=<$HOME>/.local/share/chezmoi/_home/.chezmoiignore size=66       
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/bin                                                                                                        
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/dot_byobu                                                                                                  
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/dot_sz.shrc.d                                                                                              
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/dot_sz.shrc.d/bash.post                                                                                    
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/dot_sz.shrc.d/bash.pre                                                                                     
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/dot_sz.shrc.d/zsh.post                                                                                     
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/dot_sz.shrc.d/zsh.pre                                                                                      
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_config                                                                                         
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_config/nvim                                                                                    
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_config/private__my.emacs                                                                       
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_config/private__my.emacs/private_deafult                                                       
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_config/private__my.emacs/private_deafult/private_emacs.d                                       
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_config/private__my.emacs/private_doom                                                          
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_config/private__my.emacs/private_doom/config                                                   
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_config/private__my.emacs/private_spacemacs                                                     
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_config/private__my.emacs/private_spacemacs/config.min                                          
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_config/private__my.emacs/private_spacemacs/config.std                                          
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_local                                                                                          
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_local/bin                                                                                      
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_local/private_share                                                                            
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_local/private_share/emoji                                                                      
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_local/private_share/emoji/svg
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_local/private_share/private_applications
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_local/private_share/private_fonts
2022-09-13T14:02:03-04:00 INF ReadDir component=system name=<$HOME>/.local/share/chezmoi/_home/private_dot_ssh
2022-09-13T14:02:03-04:00 INF ReadFile component=system data="<BINARY DATA>..." name=<$HOME>/.cache/chezmoi/external/08fb46f525e51c14cd1cfd7e90d6451a456e1f11cb7bcba92cbe4f3d462b776b.json.gz size=21475433
2022-09-13T14:02:14-04:00 INF HTTPRequest component=sourceState duration=10.351474747s method=GET size=21426370 status="200 OK" statusCode=200 url=https://github.com/ryanoasis/nerd-fonts/releases/download/2.2.0-RC/CodeNewRoman.zip

Output of chezmoi doctor

```console $ chezmoi doctor RESULT CHECK MESSAGE ok version v2.17.1, commit 565cbbe117746aa6bfec5f2cee20ae4cbbb5e645, built at 2022-05-30T10:24:34Z, built by goreleaser warning latest-version v2.22.1 ok os-arch linux/amd64 (Pop!_OS 22.04 LTS) ok uname Linux pop-os 5.19.0-76051900-generic #202207312230~1660780566~22.04~9d60db1 SMP PREEMPT_DYNAMIC Thu A x86_64 x86_64 x86_64 GNU/Linux ok go-version go1.18.2 (gc) ok executable ~/bin/chezmoi ok upgrade-method replace-executable ok config-file no config file found ok source-dir ~/.local/share/chezmoi is a directory ok suspicious-entries no suspicious entries ok working-tree ~/.local/share/chezmoi is a directory ok dest-dir ~ is a directory ok shell-command found /bin/bash ok shell-args /bin/bash ok cd-command found /bin/bash ok cd-args /bin/bash ok edit-command found /usr/bin/vi ok edit-args /usr/bin/vi info diff-command not set ok umask 002 ok git-command found /usr/bin/git, version 2.34.1 ok merge-command found /usr/bin/vimdiff ok age-command found /usr/bin/age, version 1.0.0 ok gpg-command found /usr/bin/gpg, version 2.2.27 info pinentry-command not set info 1password-command op not found in $PATH info bitwarden-command bw not found in $PATH info gopass-command gopass not found in $PATH info keepassxc-command keepassxc-cli not found in $PATH info keeper-command keeper not found in $PATH info keepassxc-db not set info lastpass-command lpass not found in $PATH ok pass-command found /usr/bin/pass, version 1.7.4 info vault-command vault not found in $PATH info secret-command not set ```
twpayne commented 1 year ago

Thank you very much for reporting this. #2356 fixes the immediate issue.

Do you think that chezmoi should have an "offline" mode, e.g. an extra flag like --network=false that disables all operations that need a network connection?

twpayne commented 1 year ago

1840 should also help diagnosing these sorts of problems (there's a fix in progress).

Lockszmith-GH commented 1 year ago

Thank for taking a look at this and providing a fix.

Do you think that chezmoi should have an "offline" mode, e.g. an extra flag like --network=false that disables all operations that need a network connection?

Good question. More features = more trouble... however, slow connections is something that happens. I doubt I'll encounter this again in the future myself, might be a good idea to keep tracking this and if this becomes an issue for others, then adding it would be beneficial.

If you do consider this, I would probably make it a 'level' of --dry-run, in which dry by default only skips 'touching' the local system and an offline mode where it skips 'touching' other systems as well.

In anyway, I thank you truly for making this tool and even more for maintaining it.