HTTPS + ssh access to the local Git server required for mirroring

[pp2]

Hi,

I am trying to set up a mirror from a Git repository to our local Git server. Both servers should be accessed using HTTPS, as we do not want ssh access to our local Git server.

The software versions used are as follows:

gitlab-mirrors v0.4.8 GitLab 6.9.0.rc1 9f80ab8 git 1.7.10.4

In gitlab-mirrors config.sh I only changed 2 parameters:

gitlab_url="https://our.local.git.repository.com" ssl_verify=false

Installation and configuration was as per the doc, except I set up no ssh parameters.

When I try to set up a mirror of the gitlab-mirrors project:

./add_mirror.sh --git --project-name github-gitlab-mirrors --mirror https://github.com/sag47/gitlab-mirrors.git

I get the following:

Resolving gitlab remote. Creating mirror from https://github.com/sag47/gitlab-mirrors.git Cloning into bare repository 'github-gitlab-mirrors'... remote: Reusing existing pack: 880, done. remote: Counting objects: 2, done. remote: Compressing objects: 100% (2/2), done. remote: Total 882 (delta 0), reused 2 (delta 0) Receiving objects: 100% (882/882), 148.54 KiB, done. Resolving deltas: 100% (445/445), done. Adding gitlab remote to project. Checking the mirror into gitlab. The authenticity of host 'our.local.git.repository.com (12.34.56.78)' can't be established. ECDSA key fingerprint is [...] Are you sure you want to continue connecting (yes/no)? yes

This means that the script tries to access our local Git server through ssh.

However, if I set a wrong private token in the private_token file, I instead get an immediate "UnauthorizedRequest" exception, meaning that HTTPS access is also used.

When looking at the mirrored site's config in gitlab-mirrors, i.e.:

/home/gitmirror/repositories/Mirrors/github-gitlab-mirrors/config

we can see that our local Git server is configured with an ssh URL:

[remote "gitlab"] url = git@our.local.git.repository.com.com:mirrors/github-gitlab-mirrors.git

This seeem to be due to "manage_gitlab_project.py" at line 99:

print found_project.ssh_url_to_repo

Is there a way to use only HTTPS to access the local server ? Or is this a bug or a limitation in gitlab-mirrors ?

Regards.

[samrocketman]

gitlab-mirrors currently only accesses GitLab through SSH if you're using it for API interactions (e.g. create a project on the fly using the API). This is required because gitlab-mirrors is intended to be run via cron job and can't store HTTP credentials (so SSH keys are a requirement).

However, you can skip this functionality with the --no-create option. See add_mirror.sh --help for more information. An example would be...

./add_mirror.sh --git --project-name github-gitlab-mirrors --no-create https://our.local.git.repository.com/mirrors/github-gitlab-mirrors.git --mirror https://github.com/sag47/gitlab-mirrors.git

Please note that every time you execute ./update_mirror.sh github-gitlab-mirrors or ./git-mirrors.sh you will be prompted for a password over HTTP. This is not ideal for the intended setup and I recommend you sticking to SSH.

[pp2]

Hi,

Thanks for the quick answer. Indeed, when creating the project manually through the web interface then using the "--no-create" switch to add the mirror, the command works. I use a "/home/gitmirror/.netrc" config file for bypassing HTTP login prompts so there is no interaction. However, I fail to understand the purpose of the private token if login credentials are still required. Also, why not include a "pure HTTPS" option in gitlab-mirrors ? Is it more dangerous to store a password in clear text in .netrc than a private ssh key in the .ssh directory ?

[samrocketman]

Not sure I follow for using "pure HTTPS." If your server is SSL encrypted using HTTP then it is HTTPS.

The token is for interacting with the GitLab API. --no-create forces gitlab-mirrors to not use the API and is intended for more generic non-gitlab hosting. The HTTP auth is entirely different.

Regarding the security of a password in a flat file vs an SSH private key: it does not matter. If either is compromised then one gains access to source repositories. Though, on one hand an SSH key can't be used to log into GitLab and do administrative functions (i.e. add/delete users or add/delete groups and repositories, etc). So for that reason alone I'd consider SSH better. Also, HTTPS depends on third party verification and can be man in the middle attacked quite easily if you're using self signed certificates (since you have ssl_verify=false). SSL is not buying you much security in this scenario because you're not verifying certificates. SSL can be man in the middle attacked with signed certificates if the user is using a certificate authority that your server has installed as trusted. SSH does not suffer from that issue because it verifies hosts by fingerprints or warns you if the fingerprints don't match.

I highly recommend you manage your own CA rather than use self signed certificates.

[pp2]

Hi,

By "pure HTTPS" I meant to get rid of the mix of HTTPS and ssh that shows in the initial scenario.

This should be possible if gitlab-mirrors allowed it, because I managed to do just that by doing things manually, i.e.:

• Running the initial command (i.e. without the "--no-create" switch).
• Interrupting when prompted for ssh credentials.
• Editing "repositories/Mirrors/github-gitlab-mirrors/config" to use the HTTPS URL instead of the ssh one for the "gitlab" remote.
• Running "git push gitlab" manually.

Regarding self-signed SSL certificates, this was a testing configuration and I do not know exactly yet how this works in production. However, you are right: working with self-signed certificates is a bad habit and I have successfully used your "manage your own CA" package. Thanks ! Need to install the local CA root certificate everywhere, though.

[samrocketman]

Regarding your request to interact with GitLab repositories over HTTP or SSH, I'll go ahead and label this as an enhancement or feature request. That is to say I'll work on it when I can get around to it. I also accept pull requests (PR) so if you hack on it for your needs then try to do it in a way where you can submit a PR. I'll keep this issue open until either I work on it or a PR is contributed which addresses this.

Thanks for providing me a scenario which makes this relevant. This will likely go in as a setting in config.sh for e.g.

method="SSH"

or method="HTTP" and use a combination of $gitlab_url and $gitlab_namespace to resolve the http remote. There should be a preflight() check which validates allowed input for that variable. Alternatively, I was also thinking of calling it remote_method="SSH". That's perhaps a little more descriptive.

[pp2]

Hi, I honestly don't think I'll have time to code on this but you have provided a workaround so that's OK.

[samrocketman]

@pp2 Pushing over HTTP will be a new feature in gitlab-mirrors v0.5.0. Look forward to it.