awslabs / aws-serverless-data-lake-framework

Enterprise-grade, production-hardened, serverless data lake on AWS
https://sdlf.workshop.aws/
MIT No Attribution
404 stars 137 forks source link

Stack Stuck when Creating Gitlab or Github Repo #375

Open danielxf007 opened 1 month ago

danielxf007 commented 1 month ago

Hello, when cloudformation is creating the repos using sdlf-cicd/template-cicd-sdlf-repositories.gitlab.yaml or sdlf-cicd/template-cicd-sdlf-repositories.github.yaml, the stack gets stuck on CREATE_IN_PROGRESS for the git resources, I've already activated gitlab and github extensions on cloudformation following the exmaple at https://github.com/aws-ia/cloudformation-gitlab-resource-providers/tree/main/GitLab-Projects-Project and https://github.com/aws-ia/cloudformation-github-resource-providers/tree/main/GitHub-Repositories-Repository, is there any additional configuration that I'm not aware of?

cnfait commented 1 month ago

The GitLab resource types can be a bit of a pain to use... can you try creating a simple stack with just a single GitLab project? this would make it easier to find where the issue is:

AWSTemplateFormatVersion: '2010-09-09'
Description: Shows how to create a GitLab project
Resources:
  MySampleProject:
    Type: GitLab::Projects::Project
    Properties:
      Name: my-sample-project

Let me know if you have the exact same issue with this template.

We're not completely happy with our GitLab support currently (and we consider our GitHub support not sufficiently tested yet), we're hoping to improve that later this month.

danielxf007 commented 1 month ago

Even for that one the stack gets stucked

cnfait commented 1 month ago

There are several things you can check.

--

Permissions on the GitLab access token (I do not remember the names of the permissions, but something like read_api/write_api are required)

--

The IAM role used when registering the resource type on CloudFormation registry.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/registry.html

Can you check the trust relationships of this role? does it look like this?

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "resources.cloudformation.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Usually these conditions are present too:

      "Condition": {
        "StringEquals": {
          "aws:SourceAccount": "123456789012"
        },
        "StringLike": {
          "aws:SourceArn": "arn:aws:cloudformation:us-east-1:123456789012:type/resource/Organization-Service-Resource/*"
        }

If the permissions on the role are not too broad you should try to create the stack without these conditions, check if it works, and if it does add them back one by one to understand what is causing the issue.

--

danielxf007 commented 1 month ago

I've already tried. I followed the example at https://github.com/aws-ia/cloudformation-gitlab-resource-providers/tree/main/GitLab-Projects-Project and I created the token with all permissions and I created a role with admin access and finally I did the part of the trust relationship with the source arn, yet it keeps getting stuck on CREATE_IN_PROGRESS

cnfait commented 1 month ago

Honestly this sounds like some kind of networking issue between CloudFormation and your GitLab instance, I'm not sure I can easily help you.

As I said before though, we're hoping to improve our GitLab support later this month, so maybe that will help. This is quite a bit of work though, and the days go faster than I'd like.