[Feature request] Add support for self-hosted Gitlab(CE) or GitHub Enterprise

[daannijkamp]

Feature request: add support for the self-hosted version of Gitlab (CE).

Currently the Amplify console has support for Gitlab.com, but not for the self-hosted variant. At this moment it is unfortunately a dealbreaker why we do not yet use Amplify Console for all projects/sites. As a workaround we now use the S3 and CodeCommit (by using the mirror repository function of Gitlab).

[scazan]

Is it true that we are only able to use Amplify Console if our repo is hosted externally via gitlab.com, github.com, bitbucket.org, or AWS code commit? Is this something on the roadmap? I'm curious as I would love to use Amplify!

[swaminator]

@daannijkamp @scazan A current workaround for this would be to use CodePipeline > Deploy to S3 > Setup a lambda trigger to automatically push to Amplify everytime S3 bucket is updated.

Private Git server w. CodePipeline: https://aws.amazon.com/blogs/devops/integrating-git-with-aws-codepipeline/ Amplify + S3: https://aws.amazon.com/blogs/mobile/deploy-files-s3-dropbox-amplify-console/

[anshul0915zinnia]

+1 for me

[dn-masaya-nakamura]

+1

[stugorf]

+1

[prcongithub]

+1

[raveslave]

+1

[angieyu]

+1

[jpmarques19]

+1

[kclayton925]

+1

[sonoadmin]

+1 +1 +1

[20chix]

Any update on this feature?

[ShannonMasters]

almost 2 years later, and no response from aws.. I was really looking forward to give amplify a go on a new project, but no self hosted gitlab is a deal breaker for me.

[MarcoLooy]

+1,000,000 votes

Ended up using this approach: How to Deploy a Static Website to AWS with GitLab CI

[wezside]

+1 For this feature

[Romantas]

+1

[SpolavoriBruno]

+1

[jimmyc802]

+1 - Want. Need. Please! How many upvotes do we need to get this considered?

[daannijkamp]

@swaminator @siegerts Hi, is there maybe any update you can share? Would you consider this feature request at all?

[matt-wood-ct]

+1 this needs to be a thing, what company hosts their private codebases on public git hosts?

[matt-wood-ct]

@daannijkamp @scazan A current workaround for this would be to use CodePipeline > Deploy to S3 > Setup a lambda trigger to automatically push to Amplify everytime S3 bucket is updated.

Private Git server w. CodePipeline: https://aws.amazon.com/blogs/devops/integrating-git-with-aws-codepipeline/ Amplify + S3: https://aws.amazon.com/blogs/mobile/deploy-files-s3-dropbox-amplify-console/

But you looks a bunch of valuable features like auto PR instances and in dashboard commit messages

[Oterem]

👍

[kclayton925]

I would love the Gitlab deploy feature to work with self hosted options. To get around this I used the "Deploy Without Git Provider" Then chose deploy from Amazon S3. I use the following stage in my gitlab-ci file to to push deploys to Amplify.

Deploy:
  only:
    - master
  image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest
  stage: deploy
  dependencies:
    - Build
  script:
    - aws s3 cp artifacts.zip s3://{{s3 bucket uri}}
    - aws amplify start-deployment --app-id {{app id}} --branch-name {{environment name}} --source-url s3://{{s3 bucket uri}}

[metheglin]

Gitlab works as OAuth provider. So I have researched on it and concluded that it might be achieved by a little more development on Amplify. Let me share what I tried so that Amplify developers can consider to develop this feature.

• Add a new application on self-hosted gitlab based on the following manual
  • https://docs.gitlab.com/ee/integration/oauth_provider.html#user-owned-applications
  • Set callback url according with the Amplify region
  • https://ap-northeast-1.console.aws.amazon.com/amplify/home?region=ap-northeast-1
  • Set confidential 'No'
  • Set scopes (I'm not sure which one is really required)
• Modify Application ID(client_id) corresponding with Amplify directly on database
  • This is needed because client_id is auto generated in gitlab
  • I have set the same value as Amplify Gitlab integration
  • 2db42c2cfa6ce6f6beff08fe7f2ca1d1cbc11c240e8e085802f1e2d7c53f5e81

-- Open gitlab database client
-- gitlab-rails dbconsole --database main

update oauth_applications set uid='2db42c2cfa6ce6f6beff08fe7f2ca1d1cbc11c240e8e085802f1e2d7c53f5e81' 
where uid = '--current-autogenerated-application-id-here--' and name = '--your-application-name-here--';

• Open Amplify dashboard and start creating new project
  • "New App" > "Host web app"
  • Opening browser console, Select "Gitlab" and "Continue"
  • Get the redirect request to gitlab.com and extract code in query parameter
  • Then build url as your self-hosted gitlab environment
  • https://{your-self-hosted-gitlab-here}/oauth/authorize?client_id=2db42c2cfa6ce6f6beff08fe7f2ca1d1cbc11c240e8e085802f1e2d7c53f5e81&response_type=token&redirect_uri=https://ap-northeast-1.console.aws.amazon.com/amplify/home?region=ap-northeast-1&code={extracted-code-here}

• Open the generated url
  • You'll be redirected to Amplify dashboard
  • Fetching repositories failed because the endpoint is gitlab.com not self-hosted domain. However you can find access_token is set successfully in that request

And I tried setting self-hosted repository by aws-sdk-ruby but it seems repository has strict validation and self-hosted domain was rejected. I guess it might be achived by updating the following 2 parts.

• Allow self-hosted domain for repository when creating Amplify app
• Add repository configuration so that Amplify can properly access to self-hosted gitlab api

client.create_app({
  repository: "https://{your-self-hosted-gitlab-here}/metheglin/my-next-app",
  repository_configuration: {
    type: "gitlab",
    api_prefix: "https://{your-self-hosted-gitlab-here}",
    client_id: "{application-id-generated-on-gitlab}",
  }
  ...
})

I understand there are so many concerns around communicating 3rd parties servers, but please consider once.

By the way I'm eager to use this because it's the best option for running Next.js to me. Currently it seems impossible to run SSR nodeJS application on Amplify without connecting repository.

My gitlab version: 14.4.1

[cloudgeek7]

+1 We have a customer requirement to integrate with Gitlab, can you kindly prioritise this?

[jayfry1077]

Would also like this

[Orange289]

+1

[c950291]

+1

[DarylSerrano]

+1 Can you kindly prioritize this for Bitbucket too

[allssu]

+1 ❤️ I want

[Yasanali]

Very annoying not to have this option :( +1 YES

[jakejcheng]

+1

[Tobsikrid]

+1 would need that feature right now, but hope it gets added for future generations :)

[EliaTolin]

+1

[f3lang]

Also missing that feature.

But I'd like to add, that maybe just extending it to self hosted gitlab or github enterprise might not be enough. There are a lot of different flavours, how the infrastructure for a self hosted gitlab can be set up, that can be tricky to implement a generic solution for (e.g. https access not directly possible etc.). Also just extending the existing vendor-lock-in, that is imposed here by amplify is maybe not the best move forward.

What if someone is using an entirely different setup (worst case: self-build scm solution).

So why not put a generic solution in place, that can be used by every SCM platform? You could limit it down to two settings:

• scm url with authentication (e.g. ssh key or http auth) with support for at least git (maybe also svn and mercurial or other weird stuff, but I'd like to think of git as the most commonly used scm)
• generate a callback-url for the user, that can be invoked and will trigger a pull from the repo

Essentially the existing solution does nothing else. It is just hiding those steps from the user. With the generic solution, you could just ask the user for the repo url and generate a callback url, that can be used by the user afterwards. This could also enable further improvements for deployment workflows (e.g. chatops or some other solution, that is not invented yet.).

[bruce-brookshire]

@Jay2113 do you have insight into the progress on this? I need gitlab ce access to additionally be exposed through CDK amplify. Additionally, I would expect this should be possible using a project token as opposed to a personal access token, as infra should not be tied to an explicit user entity (people leave companies etc). is there any rationale why the GitLabSourceCodeProvider cannot use a project token?

[ltickett]

I managed to write a fairly simple GitLab CI/CD deployment. See my blog post https://tickett.wordpress.com/2022/08/08/deploy-to-aws-amplify-from-gitlab-ci-cd-self-managed/

Shout if you have any questions or improvements- thanks

[bruce-brookshire]

@ltickett ill give this a try today! Thanks

[susumuf]

+1

[build3r]

Was facing the same issue with self hosted gitlab. But took an easy way to fix it. Git lab supports repository mirroring and Amplify supports AWS code commit which is AWS's source code versioning system. I just mirrored my deployment branches from Gitlab to AWS code commit and setup AWS amplify on them. Steps:

1. Goto Gitlab repo -> Settings -> Repository -> Mirroring
2. Create a Empty repo in AWS code commit
3. Use existing or generate new HTTPS git credentials
4. Add them in mirroring setup
5. Set which branches have to be mirrored and when (I keep all protected branches)
6. Go back to AWS amplify and set it up using AWS code commit.

[Yasanali]

I had to do the same. Set gitlab mirroring to code-commit.

[aentwist]

Considering the main use cases for self-hosting are potential cost savings for large scale (companies) or keeping code close (companies with security requirements)... this is a significant loss of cash and blocker for Amplify adoption?

Isn't it just adding a text input ? Assuming self-hosted GitLab is feature-complete (which is what should be assumed). No tricky setups, just a publicly-available instance.

Server URL

---

[aentwist]

The flow appears to be:

1. Visit the server URL
2. Log in to GitLab
3. For the account, add AWS Amplify as an authorized application
4. For the project, no integrations are available on the GitLab side, so create a GitLab webhook
5. Amplify can read branches, etc.
6. The webhook sends information to AWS on actions taken

This should totally work with self-hosted.

[aentwist]

The workaround without mirroring to CodeCommit is bring your own CI/CD, manually upload to S3, and use Lambda to trigger updates. This destroys half the value proposition of Amplify. Not to mention what a shame considering the handling for GitLab webhooks (the heavy lifting) has clearly already been implemented. Maybe I'm missing something but it seems like such a low fruit.

See https://docs.aws.amazon.com/amplify/latest/userguide/manual-deploys.html#amazon-s3-or-any-url

[MrMegaNova]

+1

[johnenderson]

+1

[maunzCache]

@mauerbac @manueliglesias Would this topic be something that you can drive at AWS? I do see an opportunity here to onboard a lot of customers from their on-prem setup to serverless using AWS Amplify. In combination with PrivateLink it could also be secured. Any thoughts on this?

[shixiaobao17145]

+1

[johnenderson]

Hey guys, does anyone have a workaround for deploying to AWS Amplify from a self-managed GitLab Pipeline?

[build3r]

@johnenderson You can try the mirroring to AWS code commit approach described here