feat: add BUCKET_PREFIX variable

zack-is-cool commented 6 months ago

Description

Related Issue

Fixes #44

Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Other (security config, docs update, etc)

Checklist before merging

[ ] Test, docs, adr added or updated as needed
[ ] Contributor Guide Steps followed

anthonywendt commented 6 months ago

I ran into the issue of our buckets missing that uds was added as a prefix to this new package version of gitlab when we switched to it. Is there a need for defining a prefix at all? I know we added the suffix back in the old capability. Or can we just remove the use of uds in the bucket names and only have the suffix?

zack-is-cool commented 6 months ago

For our IaC we have a pattern of adding a specific prefix and suffix for everything else, so the ability to do this works for our usecase. It was weird having everything else being named in a specific pattern, and then for these resources being restricted to using uds-.

We could default the prefix to "" if that's preferable, just like suffix, or as the code is now you could set it to be "" if you don't need/want a prefix. I'm fine with either solution.

Right now I'm working around this with 12 different overrides in my uds bundle for bucket names 😄

  - name: gitlab
    repository: ghcr.io/defenseunicorns/packages/uds/gitlab
    ref: 16.8.1-uds.4-registry1
    overrides:
      gitlab:
        gitlab:
          variables:
            - name: REGISTRY_ROLE_ARN
              description: "The ARN of the role to assume for the registry pods"
              path: registry.serviceAccount.annotations.eks\.amazonaws\.com/role-arn
            - name: SIDEKIQ_ROLE_ARN
              description: "The ARN of the role to assume for the sidekiq pods"
              path: gitlab.sidekiq.serviceAccount.annotations.eks\.amazonaws\.com/role-arn
            - name: WEBSERVICE_ROLE_ARN
              description: "The ARN of the role to assume for the web service pods"
              path: gitlab.webservice.serviceAccount.annotations.eks\.amazonaws\.com/role-arn
            - name: TOOLBOX_ROLE_ARN
              description: "The ARN of the role to assume for the toolbox pods"
              path: gitlab.toolbox.serviceAccount.annotations.eks\.amazonaws\.com/role-arn
            - name: LFS_BUCKET
              description: "The name of the S3 bucket to use for LFS storage"
              path: gitlab.global.appConfig.lfs.bucket
            - name: ARTIFACTS_BUCKET
              description: "The name of the S3 bucket to use for artifacts storage"
              path: gitlab.global.appConfig.artifacts.bucket
            - name: UPLOADS_BUCKET
              description: "The name of the S3 bucket to use for uploads storage"
              path: gitlab.global.appConfig.uploads.bucket
            - name: PACKAGES_BUCKET
              description: "The name of the S3 bucket to use for packages storage"
              path: gitlab.global.appConfig.packages.bucket
            - name: EXTERNAL_DIFFS_BUCKET
              description: "The name of the S3 bucket to use for external diffs storage"
              path: gitlab.global.appConfig.externalDiffs.bucket
            - name: TERRAFORM_STATE_BUCKET
              description: "The name of the S3 bucket to use for terraform state storage"
              path: gitlab.global.appConfig.terraformState.bucket
            - name: CI_SECURE_FILES_BUCKET
              description: "The name of the S3 bucket to use for CI secure files storage"
              path: gitlab.global.appConfig.ciSecureFiles.bucket
            - name: DEPENDENCY_PROXY_BUCKET
              description: "The name of the S3 bucket to use for dependency proxy storage"
              path: gitlab.global.appConfig.dependencyProxy.bucket
            - name: BACKUPS_BUCKET
              description: "The name of the S3 bucket to use for backups storage"
              path: gitlab.global.appConfig.backups.bucket
            - name: BACKUPS_TMP_BUCKET
              description: "The name of the S3 bucket to use for temporary backups storage"
              path: gitlab.global.appConfig.backups.tmpBucket
            - name: REGISTRY_BUCKET
              description: "The name of the S3 bucket to use for registry storage"
              path: gitlab.global.registry.bucket
            - name: PAGES_BUCKET
              description: "The name of the S3 bucket to use for pages storage"
              path: gitlab.global.pages.objectStore.bucket

justin-o12 commented 6 months ago

I ran into the issue of our buckets missing that uds was added as a prefix to this new package version of gitlab when we switched to it. Is there a need for defining a prefix at all? I know we added the suffix back in the old capability. Or can we just remove the use of uds in the bucket names and only have the suffix?

I remember we went back and forth on this during one of our first efforts to deploy SWF on EKS. Since buckets need to be globally unique, we had to use some sort of combination of suffix and prefix. I think we ended up having the suffix as optional, which meant we needed a prefix. We must have decided to stick with and hardcode uds- at that time.

Maybe the capability needs to not have an opinion and allow the consumer to use a prefix and/or a suffix.

anthonywendt commented 6 months ago

For our IaC we have a pattern of adding a specific prefix and suffix for everything else, so the ability to do this works for our usecase. It was weird having everything else being named in a specific pattern, and then for these resources being restricted to using uds-.

We could default the prefix to "" if that's preferable, just like suffix, or as the code is now you could set it to be "" if you don't need/want a prefix. I'm fine with either solution.

Right now I'm working around this with 12 different overrides in my uds bundle for bucket names 😄

That makes sense. Looks like fun haha! I don't really care. I just need to pay attention so I set up and configure our buckets correctly when we use the package.

zachariahmiller commented 6 months ago

yeah i think we need prefix and suffix. There will be times there are enforced naming conventions that have to be worked around and this better facilitates that, while also giving more customization options that can reduce the likelyhood of conflicts with globally unique names. I'm happy with this implementation (zarf var prefix default to uds-) as it wont be a breaking change.

defenseunicorns / uds-package-gitlab