Can't seem to get Terrareg to auth and pull from Github

[beezel]

I'm attempting to get this going with our on-prem Github server and running into issues. I've setup a new app, but that appears to only be for Terraform Provider stuff.

I'm trying to get Terrareg to version and publish a module that already exists in our GH org under a repo. I've tried both "https://git.corp.example.com" and "ssh://git@git.corp.example.com" as my clone URL, but it always fails with permissions issues.

I feel like I am missing a central piece of the configuration. I am supplying a private key to the "PROVIDER_SOURCES" block that is associated with the GH app, but I can't figure out where "GIT_PROVIDER_CONFIG" would be getting or using credentials? I've searched the docs high and low, but can't seem to figure out how to provide it with an SSH key or other auth method.

My .env file is as follows, redacted of course:

PROVIDER_SOURCES=[{"name": "Github", "type": "github", "base_url": "https://git.corp.example.com", "api_url": "https://git.corp.example.com/api", "client_id": "Iv1.example", "app_id": "85", "client_secret": "dexample9", "private_key_path": "/app/id_rsa_terrareg", "auto_generate_namespaces": false, "login_button_text": "Login to Terrareg GH"}]
GIT_PROVIDER_CONFIG=[{"name": "Github", "base_url": "https://git.corp.example.com/{namespace}/{module}", "clone_url": "ssh://ssh@git.corp.example.com/{namespace}/{module}.git", "browse_url": "https://git.corp.example.com/{namespace}/{module}/tree/{tag}/{path}"}]
ADMIN_AUTHENTICATION_TOKEN=RexampleB
SECRET_KEY=6example8
PUBLIC_URL=https://terrareg.corp.example.com
MIGRATE_DATABASE=True
GITHUB_API_URL=https://git.corp.example.com/api
GITHUB_URL=https://git.corp.example.com

When I try with https://git. I receive Error occurred during git clone: fatal: could not read Username for 'https://git.corp.example.com': No such device or address Command '['git', 'clone', '--single-branch', '--branch', 'v1.0.0', 'https://git.corp.example.com/iexampleg/tf-aws-module-securitygroups.git', '/tmp/tmpu7p48vql']' returned non-zero exit status 128. Cloning into '/tmp/tmpu7p48vql'... fatal: could not read Username for 'https://git.corp.example.com': No such device or address

When I try with ssh://git@git... I receive Error occurred during git clone: fatal: Could not read from remote repository. Command '['git', 'clone', '--single-branch', '--branch', 'v1.0.0', 'ssh://git@git.corp.exmple.com/example/tf-aws-module-securitygroups.git', '/tmp/tmpxuon1r7s']' returned non-zero exit status 128. Cloning into '/tmp/tmpxuon1r7s'... Warning: Permanently added 'git.corp.example.com' (ECDSA) to the list of known hosts. git@git.corp.example.com: Permission denied (publickey). fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.

I am running via container as docker run -d -p 5000:5000 --name=terrareg --env-file=/root/terrareg/.env -v /root/terrareg/id_rsa_terrareg:/app/id_rsa_terrareg ghcr.io/matthewjohn/terrareg:latest

Where am I supposed to supply the GIT_PROVIDER_CONFIG with an SSH key to allow versioning and publishing to GH?

[MatthewJohn]

Created gitlab issue: https://gitlab.dockstudios.co.uk/pub/terrareg/-/issues/540 gitlab-issue-id:540

[MatthewJohn]

Hey @beezel

Apologies for the confusion, the PROVIDER_SOURCES is purely for Terraform providers and not for modules.

But, as you've seen, GIT_PROVIDER_CONFIG is for modules. For authentication for modules, you can either use: SSH_PRIVATE_KEY (https://matthewjohn.github.io/terrareg/deployment/#ssh_private_key) or UPSTREAM_GIT_CREDENTIALS_USERNAME/UPSTREAM_GIT_CREDENTIALS_PASSWORD (https://matthewjohn.github.io/terrareg/CONFIG/#upstream_git_credentials_password).

Hope this helps

Matt

[beezel]

Thanks Matt, I'm embarrassed to have missed that! Quick followup, does this mean I'll also need a dedicated service account in addition to the Github App? I'm guessing so, but wanted to see if there was a roadmap to reduce dual auth methods into one? Thanks again!

[MatthewJohn]

Quick followup, does this mean I'll also need a dedicated service account in addition to the Github App

Yes, if you're using Github app for general authentication, then that is one thing.. but the authentication is very independent of the indexing (at least, right now). The Github authentication was kind of a side-affect of following the hashicorp-way of indexing providers.. but there are sides to it that I don't like.. in particular authenticating using other methods and relying on PAT tokens or similar, so this made me a little reluctant to jump into using it for indexing modules. But...

I'm guessing so, but wanted to see if there was a roadmap to reduce dual auth methods into one?

Yes, I think we'll end up going in that direction. The big split between modules and providers can't be there forever, so will need to find a nice way of combining them. The provider support was a little tunnel focused and haven't really had time to take another look at it and start pulling everything together.

[MatthewJohn]

@beezel Assuming that there's no further issues, I'll ensure there's a ticket for investigating the authentication flow for both modules and providers - but everything okay from your side now?

[MatthewJohn]

Have created https://gitlab.dockstudios.co.uk/pub/terrareg/-/issues/543 :)

[beezel]

Thanks for the followup, I was able to get it working just fine. Loving the util! Thanks.