junegunn / vim-plug

:hibiscus: Minimalist Vim Plugin Manager
https://junegunn.github.io/vim-plug/
MIT License
33.89k stars 1.91k forks source link

PlugUpdate fails sometimes over SSH: "Could not read from remote repository" #1093

Open alcortesm opened 3 years ago

alcortesm commented 3 years ago

I have configured git to use SSH instead of HTTP when connecting to Github:

# in my ~/.gitconig
[url "ssh://git@github.com/"]
    insteadOf = https://github.com/

I use gpg-agent to remember my private SSH key for a certain amount of hours, so I only have to write my passphrase once a day to access my SSH key:

; # I can do this without having to write my passphrase again and again
; ssh -T git@github.com
Hi alcortesm! You've successfully authenticated, but GitHub does not provide shell access.

But when I tell vim-plug to check for updates I get inconsistent behaviour:

Every time I try to PlugUpdate, different plugins show different behaviour from the list above and I haven't been able to find any patterns. I use 12 plugins, every time I try to update them I get 1-3 errors and 1-2 pinentry windows popups.

The problem only happens if I configure git to use SSH instead of HTTP, if I disable that, all plugins are correctly updated.


VIM - Vi IMproved 8.1 (2018 May 18, compiled Jun 15 2019 16:41:15)
Included patches: 1-875, 878, 884, 948, 1046, 1365-1368, 1382, 1401
Modified by team+vim@tracker.debian.org
Compiled by team+vim@tracker.debian.org
Huge version with GTK2 GUI.  Features included (+) or not (-):
+acl               +extra_search      +mouse_netterm     +tag_old_static
+arabic            +farsi             +mouse_sgr         -tag_any_white
+autocmd           +file_in_path      -mouse_sysmouse    +tcl
+autochdir         +find_in_path      +mouse_urxvt       +termguicolors
-autoservername    +float             +mouse_xterm       +terminal
+balloon_eval      +folding           +multi_byte        +terminfo
+balloon_eval_term -footer            +multi_lang        +termresponse
+browse            +fork()            -mzscheme          +textobjects
++builtin_terms    +gettext           +netbeans_intg     +textprop
+byte_offset       -hangul_input      +num64             +timers
+channel           +iconv             +packages          +title
+cindent           +insert_expand     +path_extra        +toolbar
+clientserver      +job               +perl              +user_commands
+clipboard         +jumplist          +persistent_undo   +vartabs
+cmdline_compl     +keymap            +postscript        +vertsplit
+cmdline_hist      +lambda            +printer           +virtualedit
+cmdline_info      +langmap           +profile           +visual
+comments          +libcall           -python            +visualextra
+conceal           +linebreak         +python3           +viminfo
+cryptv            +lispindent        +quickfix          +vreplace
+cscope            +listcmds          +reltime           +wildignore
+cursorbind        +localmap          +rightleft         +wildmenu
+cursorshape       +lua               +ruby              +windows
+dialog_con_gui    +menu              +scrollbind        +writebackup
+diff              +mksession         +signs             +X11
+digraphs          +modify_fname      +smartindent       -xfontset
+dnd               +mouse             +startuptime       +xim
-ebcdic            +mouseshape        +statusline        +xpm
+emacs_tags        +mouse_dec         -sun_workshop      +xsmp_interact
+eval              +mouse_gpm         +syntax            +xterm_clipboard
+ex_extra          -mouse_jsbterm     +tag_binary        -xterm_save
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  system gvimrc file: "$VIM/gvimrc"
    user gvimrc file: "$HOME/.gvimrc"
2nd user gvimrc file: "~/.vim/gvimrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
    system menu file: "$VIMRUNTIME/menu.vim"
  fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_GTK  -pthread -I/usr/include/gtk-2.0 -I/usr/lib/x86_64-linux-gnu/gtk-2.0/include -I/usr/inEvery time I update the plugins each plugin show
clude/gio-unix-2.0 -I/usr/include/cairo -I/usr/include/pango-1.0 -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/pango-1.0 -I/usr/include/harfbuzz -I/usr/include/pango-1Every time I update the plugins each plugin show
.0 -I/usr/include/fribidi -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/uuid -I/usr/include/freetype2 -I/usr/include/libpng16 -Wdate-time  -g -O2 -fdebug-prefix-map=/build/vim-4Pursk/vim-8.1.0875=. -fstack-protector-strong -Wformat -Werror=format-security -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1       
Linking: gcc   -L. -Wl,-z,relro -Wl,-z,now -fstack-protector -rdynamic -Wl,-export-dynamic -Wl,-E  -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -o vim   -lgtk-x11-2.0 -lgdk-x11-2.0 -lpangocairo-1.0 -latk-1.0 -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lpangoft2-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lfontconfig -lfreetype -lSM -lICE -lXpm -lXt -lX11 -lXdmcp -lSM -lICE  -lm -ltinfo -lnsl  -lselinux  -lacl -lattr -lgpm -ldl  -L/usr/lib -llua5.2 -Wl,-E  -fstack-protector-strong -L/usr/local/lib  -L/usr/lib/x86_64-linux-gnu/perl/5.28/CORE -lperl -ldl -lm -lpthread -lcrypt  -L/usr/lib/python3.7/config-3.7m-x86_64-linux-gnu -lpython3.7m -lcrypt -lpthread -ldl -lutil -lm -L/usr/lib/x86_64-linux-gnu -ltcl8.6 -ldl -lz -lpthread -lm -lruby-2.5 -lpthread -lgmp -ldl -lcrypt -lm     
alcortesm commented 3 years ago

The problem is still there if I use :PlugUpdate --sync instead of just :PlugUpdate.

Some more details about my system:

; git --version
git version 2.20.1

; gpg-agent --version
gpg-agent (GnuPG) 2.2.12
libgcrypt 1.8.4
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

I can reproduce the problem with a minimal .vimrc like this:

; cat .vimrc
call plug#begin('~/.vim/plugged')

Plug 'tpope/vim-fugitive'
Plug 'preservim/nerdtree'
Plug 'AndrewRadev/splitjoin.vim'
Plug 'fatih/vim-go', { 'do': ':GoUpdateBinaries' }
Plug 'christoomey/vim-tmux-navigator'
Plug 'SirVer/ultisnips'
Plug 'junegunn/fzf', { 'do': { -> fzf#install() } }
Plug 'junegunn/fzf.vim'
Plug 'pedrohdz/vim-yaml-folds'
Plug 'tpope/vim-obsession'
Plug 'michaeljsmith/vim-indent-object'

call plug#end()

" I use the fish shell, so just in case I will leave this here.
set shell=/bin/sh
dmcinnes commented 2 years ago

I'm having a similar issue with :PlugUpdate I also rewrite my github urls:

[url "git@github.com:"]
   insteadOf = https://github.com/
fatal: Could not read from remote repository.

Doesn't happen on all plugins, but an example of one that fails is https://github.com/tpope/vim-rhubarb. If I remove the rewrite from my github config, :PlugUpdate works fine.

gwd999 commented 2 years ago

Have the same issue with some packages updating via ssh some not of which some seem to require https; all very mysterious - could that be related to passphrase being required but then why does modifying my .gitconfig have an effect on the "updateability"? tbh I am a bit lost here

michaelxor commented 1 year ago

FYI to folks hitting this -- I believe Github is rate-limiting pulls over SSH. Like others, I have the following in my ~/.gitconfig:

[url "ssh://git@github.com/"]
    insteadOf = https://github.com/

And, for completeness, I have the following in my ~/.ssh/config:

Host github.com
    AddKeysToAgent yes
    UseKeychain yes
    IdentityFile ~/path/to/key

I am experiencing similar behavior to @alcortesm where upon :PlugUpdate (or :PlugUpdate --sync), some plugins update as intended while others fail with this error:

fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists.

It turns out, the same things happens if try to invoke git pull directly using a bash for loop:

❯ cd ~/.vim/plugged
❯ for repo in $(ls .); do echo $repo; pushd $repo > /dev/null; git pull; popd > /dev/null; printf "\n\n"; done

When running the above shell command, I observed that the first ~25 repos updated nominally, then the rest (I have 49 total) failed with the same error. I was able to reproduce this several times, running the above a few minutes apart each time. If I instead run git pull --verbose, the error becomes:

ssh_exchange_identification: read: Connection reset by peer
fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists.

If I insert a sleep 3 after each git pull, however, all repos will update successfully! So my empirical observation is that Github is likely limiting clients, and even iterating through this stuff synchronously is not necessarily going to work unless the pulls are spaced out.

This SSH rate-limiting behavior can be observed trivially with the following:

❯ for i in {1..50}; do echo $i; date; ssh -T git@github.com; done
1
Fri Sep 16 11:17:08 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
2
Fri Sep 16 11:17:09 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
3
Fri Sep 16 11:17:10 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
4
Fri Sep 16 11:17:11 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
5
Fri Sep 16 11:17:12 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
6
Fri Sep 16 11:17:14 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
7
Fri Sep 16 11:17:15 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
8
Fri Sep 16 11:17:16 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
9
Fri Sep 16 11:17:17 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
10
Fri Sep 16 11:17:18 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
11
Fri Sep 16 11:17:20 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
12
Fri Sep 16 11:17:21 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
13
Fri Sep 16 11:17:22 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
14
Fri Sep 16 11:17:23 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
15
Fri Sep 16 11:17:25 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
16
Fri Sep 16 11:17:26 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
17
Fri Sep 16 11:17:27 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
18
Fri Sep 16 11:17:28 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
19
Fri Sep 16 11:17:29 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
20
Fri Sep 16 11:17:31 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
21
Fri Sep 16 11:17:32 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
22
Fri Sep 16 11:17:34 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
23
Fri Sep 16 11:17:35 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
24
Fri Sep 16 11:17:37 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
25
Fri Sep 16 11:17:39 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
26
Fri Sep 16 11:17:40 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
27
Fri Sep 16 11:17:42 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
28
Fri Sep 16 11:17:43 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
29
Fri Sep 16 11:17:45 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
30
Fri Sep 16 11:17:46 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
31
Fri Sep 16 11:17:48 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
32
Fri Sep 16 11:17:49 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
33
Fri Sep 16 11:17:50 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
34
Fri Sep 16 11:17:52 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
35
Fri Sep 16 11:17:54 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
36
Fri Sep 16 11:17:55 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
37
Fri Sep 16 11:17:57 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
38
Fri Sep 16 11:17:58 PDT 2022
Hi michaelxor! You've successfully authenticated, but GitHub does not provide shell access.
39
Fri Sep 16 11:18:00 PDT 2022
ssh_exchange_identification: read: Connection reset by peer
40
Fri Sep 16 11:18:00 PDT 2022
ssh_exchange_identification: read: Connection reset by peer
41
Fri Sep 16 11:18:00 PDT 2022
ssh_exchange_identification: read: Connection reset by peer
42
Fri Sep 16 11:18:00 PDT 2022
ssh_exchange_identification: read: Connection reset by peer
43
Fri Sep 16 11:18:01 PDT 2022
ssh_exchange_identification: read: Connection reset by peer
44
Fri Sep 16 11:18:01 PDT 2022
ssh_exchange_identification: read: Connection reset by peer
45
Fri Sep 16 11:18:01 PDT 2022
ssh_exchange_identification: read: Connection reset by peer
46
Fri Sep 16 11:18:01 PDT 2022
ssh_exchange_identification: read: Connection reset by peer
47
Fri Sep 16 11:18:02 PDT 2022
ssh_exchange_identification: read: Connection reset by peer
48
Fri Sep 16 11:18:02 PDT 2022
ssh_exchange_identification: read: Connection reset by peer
49
Fri Sep 16 11:18:02 PDT 2022
ssh_exchange_identification: read: Connection reset by peer
50
Fri Sep 16 11:18:02 PDT 2022
ssh_exchange_identification: read: Connection reset by peer

Seems like a nice update here would be some attempt to detect the rate limit condition and back off / retry plugin updates that failed for this reason. The naive fix that worked for me was "wait 3 seconds between each pull".

michaelxor commented 1 year ago

Follow up to my comment above - I followed this article from Puppet on re-using SSH connections in an effort to mitigate the rate-limiting by creating fewer connections during plugin update. I updated my ~/.ssh/config to the following:

Host github.com
    ControlMaster auto
    ControlPath ~/.ssh/sockets/%r@%h-%p
    ControlPersist 600
    AddKeysToAgent yes
    UseKeychain yes
    IdentityFile ~/path/to/key

(and, as instructed in the linked article, manually created the ~/.ssh/sockets directory).

While this did not completely mitigate the issue, it did seem to help and fewer plugin updates failed when I next ran :PlugUpdate. I was then able to use the 'R' to retry failed plugin updates once to capture the rest of the plugins. YMMV

danielcbaldwin commented 1 year ago

@michaelxor I was running into this same issue, your fix of reusing ssh connections along with setting the job concurrency of vim-plug to 4 or less solved the problem for me. Thanks!

jgmchan commented 2 months ago

Some additional information when re-using the SSH connections, it looks like Github is just using the default maximum of 10 concurrent sessions in OpenSSH.

From man sshd_config

MaxSessions
    Specifies the maximum number of open shell, login or subsystem (e.g. sftp) sessions permitted per network connection.  
    Multiple sessions may be established by clients that support connection multiplexing.  Setting MaxSessions to 1 will 
    effectively disable session multiplexing, whereas setting it to 0 will prevent all shell, login and subsystem sessions 
    while still permitting forwarding.  The default is 10.

When I set the job currency to 10 (:PlugUpdate10) everything is working fine, any more and I get the error again.

I wonder if it is possible to have the default job concurrency set to 10 as well or have a vim-plug configuration where I can set the concurrency myself in .vimrc?

junegunn commented 2 months ago

have the default job concurrency set to 10

let g:plug_threads = 10

[!NOTE] The name is a bit misleading because we're not really using "threads" in the recent versions of Vim. But we used to use threads in Ruby and Python installer that was used before Vim 8 or Neovim introduced the concept of "jobs", and the name stuck.