caddy-git unable to clone repos with non-standard port URL

gganeshan commented 7 years ago

caddy-git plugin is no longer able to clone repos with non-standard port URL. This was working a few weeks back and could have been introduced by this change

git {
        repo        ssh://git@stash.abc.net:7999/project/repo.git
        key         ~/.ssh/id_rsa
}

Multiple issues:

By default, the plugin tries to connect at port 22.

error

Activating privacy features... done.
Cloning into '/var/www'...
ssh: connect to host stash.abc.net port 22: Connection refused
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
2017/09/07 15:19:58 exit status 128

If the port is specified in ~/.ssh/cong then it is able to connect to the server but still cannot clone the repo

ssh config

Host stash.abc.net
    StrictHostKeyChecking no
    port 7999

error

Activating privacy features... done.
Cloning into '/var/www'...
Repository not found
The requested repository does not exist, or you do not have permission to access it.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

I tried the workaround suggested here but it did not work for me.

Note: I am able to clone the same URL manually.

gganeshan commented 7 years ago

Guess, the git command being run by the plugin is:

git clone ssh://git@stash.abc.net/7999/project/repo.git
Cloning into 'repo'...
Repository not found
The requested repository does not exist, or you do not have permission to access it.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

:eyes: how :7999 got converted to /7999

gganeshan commented 7 years ago

Disregard the above comment. It was just a guess :disappointed: .

abiosoft commented 7 years ago

Let me get this right, what is the log output. Is :7999 converted to /7999 ?

gganeshan commented 7 years ago

Let me get this right, what is the log output. Is :7999 converted to /7999 ?

This was just a guess. The log output is not verbose enough to determine the URL being used.

gganeshan commented 7 years ago

@abiosoft any updates on this issue?? Were you able to recreate it??

mdcollins05 commented 7 years ago

I'm having the same issue where the port is being ignored and not passed to ssh-keyscan as an argument and when it attempts to clone it tries to connect on port 22 regardless of the actual setting.

It was working in the past but now it stopped when I updated on Sept 11th.

Edit: While I was debugging, I noticed despite the configuration having ssh:// in front of the repo URL, it's not passed to the git clone command.

Additionally, the output of ssh-keyscan was put into the known_hosts and tmp_hosts files even when the domain wasn't able to be looked up due to unrelated issues I was having. It also seems that some of the contents in the known_hosts file is duplicated each startup.

This caused the known_hosts file to grow to about 2.5MB until I removed it. It consisted of approximately 21,000 lines, mainly with the text getaddrinfo aeiorjheriohjriot.com: Name or service not known where the domain name was my actual domain.

Unfortunately, while looking at my server logs, I lost the scroll back in my terminal for the above mentioned issues and can no longer reproduce them with the exception of the duplicated entries in my known_hosts file and the missing port argument on the ssh-keyscan command.

Sorry, one last edit: I think this line is the cause of the duplicate entries in the known_hosts file: https://github.com/abiosoft/caddy-git/blob/master/scripts.go#L119

It seems to me that the output of the known_hosts file is being sorted and duplicate lines removed. That is put into the tmp_hosts file but then we are appending the contents of the tmp_hosts into the known_hosts file, causing the duplicates.

abiosoft commented 7 years ago

Thanks for digging into that. I should be able to look into it this weekend.

gganeshan commented 7 years ago

Thanks for digging into that. I should be able to look into it this weekend.

Any updates @abiosoft ?

gganeshan commented 7 years ago

Additionally, the output of ssh-keyscan was put into the known_hosts and tmp_hosts files even when the domain wasn't able to be looked up due to unrelated issues I was having. It also seems that some of the contents in the known_hosts file is duplicated each startup.

I saw the same behavior in my container.

mdcollins05 commented 6 years ago

Any luck on working through this issue @abiosoft ? Unfortunately, this breaks one of my web servers and prevents it from starting.

gganeshan commented 6 years ago

Any updates on this @abiosoft ??

arahmanhamdy commented 6 years ago

any updates @abiosoft ?

pixie79 commented 5 years ago

This bug appears to still be present 31 user1 0:00 bash /tmp/caddy265241523 -i /home/user1/.ssh/id_rsa clone -b develop git@gitlab.example.uk:2222/cop/policy.git /repo 33 user1 0:00 /usr/bin/git clone -b develop git@gitlab.example.uk:2222/cop/policy.git /repo 36 user1 0:00 bash /tmp/.git_ssh.31 git@gitlab.example.uk git-upload-pack '2222/cop/policy.git' 37 user1 0:00 ssh -i /home/user1/.ssh/id_rsa git@gitlab.example.uk git-upload-pack '2222/cop/policy.git'

As you can see the port 2222 is being added to the path and not added as a -p port variable to the command.

abiosoft / caddy-git