search

EmbarkStudios / k8s-buildkite-plugin

Run any buildkite build step as a Kubernetes Job
https://embark.dev
Apache License 2.0
49 stars 19 forks source link

`k8s-buildkite-agent` built on CI has missing executables #56

Closed loeffel-io closed 1 year ago

loeffel-io commented 1 year ago

Hello,

i try to configure the buildkite helm chart. It creates those two base64 secrets (https://github.com/buildkite/charts/blob/master/stable/agent/templates/secret.yaml#L12): 

kubectl describe secret buildkite-agent  -n buildkite

Name:         buildkite-agent
Namespace:    buildkite
Labels:       app=agent
              app.kubernetes.io/managed-by=Helm
              chart=agent-0.6.1
              heritage=Helm
              release=buildkite
Annotations:  meta.helm.sh/release-name: buildkite
              meta.helm.sh/release-namespace: buildkite

Type:  Opaque

Data
====
agent-ssh:    80 bytes
agent-token:  50 bytes

How to configure the Plugin now? I am pretty confused about all those different options like secret-name and default-secret-name

This is my current configuration

      - plugins:
          - EmbarkStudios/k8s:
              image: "gcr.io/bazel-public/bazel:6.0.0"
              entrypoint: ""
              shell: [ "sh", "-e", "-c" ]
              service-account-name: "global-base-production"
              secret-name: "buildkite-agent"
              agent-token-secret-key: "agent-token"
              git-ssh-secret-key: "agent-ssh"

Would really like to get some quick help 🙏 ❤️

loeffel-io commented 1 year ago

@tgolsson i finally made it. this solves the problem (adding $'\n' at the end, gcloud trims everthing) 

buildkite_private_ssh_key=$(CLOUDSDK_CONFIG="$chdir/$CLOUDSDK_CONFIG" $gcloud \
  secrets versions access latest \
  --secret="buildkite-private-ssh-key-$env" \
  --project="$gcloud_project_id")$'\n'

but there are some bugs using your k8s-buildkite-agent image:

it looks like the job is starting and it runs bazel but at the same time there are no logs and the output gets stucked at bootstrap container

whats now wrong?

Bildschirm­foto 2023-02-03 um 16 19 05 Bildschirm­foto 2023-02-03 um 16 14 59
loeffel-io commented 1 year ago

here the full log, everything looks great downloaded-logs-20230203-162218.json.zip

loeffel-io commented 1 year ago

current setup: 

steps:
  - group: "Global"
    key: "global"
    steps:
      - plugins:
          - EmbarkStudios/k8s#6b36fe4f6b770cdb97fd420b50cc94cc1c0bcbce:
              image: "gcr.io/bazel-public/bazel:6.0.0"
              entrypoint: ""
              shell: [ "sh", "-e", "-c" ]
              service-account-name: "global-base-production"
              secret-name: "buildkite-agent"
              agent-token-secret-key: "agent-token"
              git-ssh-secret-key: "agent-ssh"
              init-image: "embarkstudios/k8s-buildkite-agent@sha256:1d88791315ed6b0b49a64055bc71c5a9a0b1953e387f99d25299ed06ccea5dbd"
        label: "global"
        command:
          - bazel test --remote_cache=$GOOGLE_BUCKET_PRODUCTION --google_credentials=$GOOGLE_CREDENTIALS_PRODUCTION //...
          - bazel build --remote_cache=$GOOGLE_BUCKET_PRODUCTION --google_credentials=$GOOGLE_CREDENTIALS_PRODUCTION //...
tgolsson commented 1 year ago

So it's working? I'm having a hard time parsing the logs in JSON format :P

loeffel-io commented 1 year ago

yes, my build is working, but buildkite.com is not showing any results and gets stuck - so this is not working and for sure belongs to the plugin

here are some logs of everything pass downloaded-logs-20230203-115521.json.zip

here the job is done successfully

Bildschirm­foto 2023-02-03 um 16 37 16

and this is all what i can see (no log output and stuck there forever):

Bildschirm­foto 2023-02-03 um 16 14 59
loeffel-io commented 1 year ago

way more easy to read 😄

downloaded-logs-20230203-164347.csv

tgolsson commented 1 year ago

Interesting. It looks like your bootstrap container ran for a very short time; so I'm wondering if the logs-tailer somehow gets stuck waiting for it to finish... There's a few loops in the logs tailing part. I've tried a few variants of the command but can't repro yet.

loeffel-io commented 1 year ago

this is all i do

steps:
  - group: "Global"
    key: "global"
    steps:
      - plugins:
          - EmbarkStudios/k8s#6b36fe4f6b770cdb97fd420b50cc94cc1c0bcbce:
              image: "gcr.io/bazel-public/bazel:6.0.0"
              entrypoint: ""
              shell: [ "sh", "-e", "-c" ]
              service-account-name: "global-base-production"
              secret-name: "buildkite-agent"
              agent-token-secret-key: "agent-token"
              git-ssh-secret-key: "agent-ssh"
              init-image: "embarkstudios/k8s-buildkite-agent@sha256:1d88791315ed6b0b49a64055bc71c5a9a0b1953e387f99d25299ed06ccea5dbd"
        label: "global"
        command:
          - bazel test //...

plain untouched gke cluster

tgolsson commented 1 year ago

Yeah; nothing there looks wrong. If you cancel and rerun, does the issue get reproduced? Or is it random?

loeffel-io commented 1 year ago

its everytime the same, also with this config

steps:
  - name: shellcheck hooks
    command:
      - echo "Running shellcheck"
      - shellcheck hooks/*
    plugins:
      - EmbarkStudios/k8s#6b36fe4f6b770cdb97fd420b50cc94cc1c0bcbce:
          image: koalaman/shellcheck-alpine
          default-secret-name: buildkite-k8s-plugin
          use-agent-node-affinity: true
Bildschirm­foto 2023-02-03 um 17 23 28

thank you btw so much that you are still here ❤️

tgolsson commented 1 year ago

That is interesting. I've added a commit 1e7f666c6a6752a65c4ae22d28b9150a82af0414 that adds some logs so we can see where it gets stuck - can you try with that instead of 6b...?

loeffel-io commented 1 year ago

sure :) sec

loeffel-io commented 1 year ago

downloaded-logs-20230203-173436.csv

loeffel-io commented 1 year ago
Bildschirm­foto 2023-02-03 um 17 35 09
tgolsson commented 1 year ago

What about edaafe75e52d5aaa415941830a2de95514e3dd38 ? Added some more detail - I'm wondering if you're hitting the timeout case if your API-server is under heavy load. The default is just 5 seconds.

loeffel-io commented 1 year ago
Bildschirm­foto 2023-02-03 um 17 41 56

downloaded-logs-20230203-174343.csv

tgolsson commented 1 year ago

That sounds like you need to add the logs GET permission to the role of buildkite-agent, whatever that is. I'm guessing you've somewhere provided the permission of creating jobs already?

loeffel-io commented 1 year ago

I just use the buildkite-helm-chart config with one added property: 

rbac:
  create: true
  role:
    rules:
      - apiGroups:
          - ""
        resources:
          - namespaces
        verbs:
          - watch
          - list
      - apiGroups:
          - ""
        resources:
          - secrets
        verbs:
          - get
          - watch
          - list
      - apiGroups:
          - ""
          - extensions
          - apps
          - batch
        resources:
          - pods
          - replicasets
          - replicationcontrollers
          - statefulsets
          - deployments
          - daemonsets
          - jobs
          - cronjobs
        verbs:
          - get
          - delete
          - watch
          - list
          - update
          - create # <- added
      - apiGroups:
          - ""
        resources:
          - configmaps
          - secrets
        verbs:
          - get
          - create
          - update

what would you change here?

tgolsson commented 1 year ago

I think adding pods/logs below podsshould be sufficient.

loeffel-io commented 1 year ago

like 

rbac:
  create: true
  role:
    rules:
      - apiGroups:
          - ""
        resources:
          - namespaces
        verbs:
          - watch
          - list
      - apiGroups:
          - ""
        resources:
          - secrets
        verbs:
          - get
          - watch
          - list
      - apiGroups:
          - ""
          - extensions
          - apps
          - batch
        resources:
          - pods
          - pods/logs # <-- here? 
          - replicasets
          - replicationcontrollers
          - statefulsets
          - deployments
          - daemonsets
          - jobs
          - cronjobs
        verbs:
          - get
          - delete
          - watch
          - list
          - update
          - create # <- added
      - apiGroups:
          - ""
        resources:
          - configmaps
          - secrets
        verbs:
          - get
          - create
          - update
tgolsson commented 1 year ago

That looks good. Some sources say you need pods/logs, some say pods/log. Both might work.

loeffel-io commented 1 year ago

pods/log worked

now i get this:

Bildschirm­foto 2023-02-03 um 18 04 20
tgolsson commented 1 year ago

That's fine! The loop is meant to cover that, it was just hiding the actual failure too. I'd go back to the original commit (before we started adding logs) and try with that now.

loeffel-io commented 1 year ago

just give me some mins for meal

tgolsson commented 1 year ago

Yeah no worries! I'm leaving for the weekend now so will be a bit less responsive :-) I'll see if I can find to clean up and release the bugfixes, and file some bugs for RBAC diagnostics.

loeffel-io commented 1 year ago

with 1e7f666c6a6752a65c4ae22d28b9150a82af0414 i can finally see some logs 🎉

thank you so much and enjoy your weekend! ❤️

i think it would be really helpful when we mention in the docs that pods/log permissions are required etc 

~~~ Preparing plugins
# Plugin "github.com/EmbarkStudios/k8s-buildkite-plugin#1e7f666c6a6752a65c4ae22d28b9150a82af0414" already checked out (1e7f666)
~~~ Preparing working directory
$ cd /buildkite/builds/buildkite-agent-6b59766cd4-ft4cj-1/mindful/global-base
# Host "github.com" already in list of known hosts at "/root/.ssh/known_hosts"
$ git remote set-url origin git@github.com:mindful-hq/global-base.git
$ git clean -ffxdq
$ git fetch -v --prune -- origin d7ed53a809c877cab0d827dfbfc0caea49613a26
From github.com:mindful-hq/global-base

 * branch            d7ed53a809c877cab0d827dfbfc0caea49613a26 -> FETCH_HEAD

$ git checkout -f d7ed53a809c877cab0d827dfbfc0caea49613a26
HEAD is now at d7ed53a test: buildkite

# Cleaning again to catch any post-checkout changes
$ git clean -ffxdq
# Checking to see if Git data needs to be sent to Buildkite
$ buildkite-agent meta-data exists buildkite:git:commit
~~~ Running plugin k8s command hook
$ /buildkite/plugins/github-com-EmbarkStudios-k8s-buildkite-plugin-1e7f666c6a6752a65c4ae22d28b9150a82af0414/hooks/command
--- :kubernetes: Starting Kubernetes Job

-rw------- 1 root root 2405 Feb  3 17:32 /tmp/job-env-01861856-15d5-48af-8292-30d54af5f5193990684667

job.batch/global-base-88-tyxog3af created

Timeout: 36000s

--- :kubernetes: Running image: gcr.io/bazel-public/bazel:6.0.0

Pod is running: global-base-88-tyxog3af-l54rq

--- :kubernetes: bootstrap container

Polling for pod to start

Starting to tail logs...

+ git config --global credential.helper 'store --file=/secrets/git-credentials'

+ [[ -f /secrets/ssh-key ]]

++ ssh-agent -s

+ eval 'SSH_AUTH_SOCK=/tmp/ssh-XXXXXXPBochi/agent.8; export SSH_AUTH_SOCK;

SSH_AGENT_PID=9; export SSH_AGENT_PID;

echo Agent pid 9;'

++ SSH_AUTH_SOCK=/tmp/ssh-XXXXXXPBochi/agent.8

++ export SSH_AUTH_SOCK

++ SSH_AGENT_PID=9

++ export SSH_AGENT_PID

++ echo Agent pid 9

Agent pid 9

+ ssh-add -k /secrets/ssh-key

Identity added: /secrets/ssh-key (/secrets/ssh-key)

+ [[ -d /local ]]

+ cp /usr/local/bin/buildkite-agent /local/

+ exec /usr/local/bin/buildkite-agent bootstrap --experiment=git-mirrors --git-mirrors-path=/git-mirrors --ssh-keyscan --command true

~~~ Preparing plugins

# Plugin "github.com/EmbarkStudios/k8s-buildkite-plugin" will be checked out to "/buildkite/plugins/github-com-EmbarkStudios-k8s-buildkite-plugin-1e7f666c6a6752a65c4ae22d28b9150a82af0414"

$ cd /buildkite/plugins/github-com-EmbarkStudios-k8s-buildkite-plugin-1e7f666c6a6752a65c4ae22d28b9150a82af0414

# Switching to the plugin directory

$ git clone -v -- https://github.com/EmbarkStudios/k8s-buildkite-plugin .

Cloning into '.'...

POST git-upload-pack (175 bytes)

POST git-upload-pack (gzip 1702 to 833 bytes)

remote: Enumerating objects: 709, done.

remote: Counting objects:   0% (1/245)
remote: Counting objects:   1% (3/245)
remote: Counting objects:   2% (5/245)
remote: Counting objects:   3% (8/245)
remote: Counting objects:   4% (10/245)
remote: Counting objects:   5% (13/245)
remote: Counting objects:   6% (15/245)
remote: Counting objects:   7% (18/245)
remote: Counting objects:   8% (20/245)
remote: Counting objects:   9% (23/245)
remote: Counting objects:  10% (25/245)
remote: Counting objects:  11% (27/245)
remote: Counting objects:  12% (30/245)
remote: Counting objects:  13% (32/245)
remote: Counting objects:  14% (35/245)
remote: Counting objects:  15% (37/245)
remote: Counting objects:  16% (40/245)
remote: Counting objects:  17% (42/245)
remote: Counting objects:  18% (45/245)
remote: Counting objects:  19% (47/245)
remote: Counting objects:  20% (49/245)
remote: Counting objects:  21% (52/245)
remote: Counting objects:  22% (54/245)
remote: Counting objects:  23% (57/245)
remote: Counting objects:  24% (59/245)
remote: Counting objects:  25% (62/245)
remote: Counting objects:  26% (64/245)
remote: Counting objects:  27% (67/245)
remote: Counting objects:  28% (69/245)
remote: Counting objects:  29% (72/245)
remote: Counting objects:  30% (74/245)
remote: Counting objects:  31% (76/245)
remote: Counting objects:  32% (79/245)
remote: Counting objects:  33% (81/245)
remote: Counting objects:  34% (84/245)
remote: Counting objects:  35% (86/245)
remote: Counting objects:  36% (89/245)
remote: Counting objects:  37% (91/245)
remote: Counting objects:  38% (94/245)
remote: Counting objects:  39% (96/245)
remote: Counting objects:  40% (98/245)
remote: Counting objects:  41% (101/245)
remote: Counting objects:  42% (103/245)
remote: Counting objects:  43% (106/245)
remote: Counting objects:  44% (108/245)
remote: Counting objects:  45% (111/245)
remote: Counting objects:  46% (113/245)
remote: Counting objects:  47% (116/245)
remote: Counting objects:  48% (118/245)
remote: Counting objects:  49% (121/245)
remote: Counting objects:  50% (123/245)
remote: Counting objects:  51% (125/245)
remote: Counting objects:  52% (128/245)
remote: Counting objects:  53% (130/245)
remote: Counting objects:  54% (133/245)
remote: Counting objects:  55% (135/245)
remote: Counting objects:  56% (138/245)
remote: Counting objects:  57% (140/245)
remote: Counting objects:  58% (143/245)
remote: Counting objects:  59% (145/245)
remote: Counting objects:  60% (147/245)
remote: Counting objects:  61% (150/245)
remote: Counting objects:  62% (152/245)
remote: Counting objects:  63% (155/245)
remote: Counting objects:  64% (157/245)
remote: Counting objects:  65% (160/245)
remote: Counting objects:  66% (162/245)
remote: Counting objects:  67% (165/245)
remote: Counting objects:  68% (167/245)
remote: Counting objects:  69% (170/245)
remote: Counting objects:  70% (172/245)
remote: Counting objects:  71% (174/245)
remote: Counting objects:  72% (177/245)
remote: Counting objects:  73% (179/245)
remote: Counting objects:  74% (182/245)
remote: Counting objects:  75% (184/245)
remote: Counting objects:  76% (187/245)
remote: Counting objects:  77% (189/245)
remote: Counting objects:  78% (192/245)
remote: Counting objects:  79% (194/245)
remote: Counting objects:  80% (196/245)
remote: Counting objects:  81% (199/245)
remote: Counting objects:  82% (201/245)
remote: Counting objects:  83% (204/245)
remote: Counting objects:  84% (206/245)
remote: Counting objects:  85% (209/245)
remote: Counting objects:  86% (211/245)
remote: Counting objects:  87% (214/245)
remote: Counting objects:  88% (216/245)
remote: Counting objects:  89% (219/245)
remote: Counting objects:  90% (221/245)
remote: Counting objects:  91% (223/245)
remote: Counting objects:  92% (226/245)
remote: Counting objects:  93% (228/245)
remote: Counting objects:  94% (231/245)
remote: Counting objects:  95% (233/245)
remote: Counting objects:  96% (236/245)
remote: Counting objects:  97% (238/245)
remote: Counting objects:  98% (241/245)
remote: Counting objects:  99% (243/245)
remote: Counting objects: 100% (245/245)
remote: Counting objects: 100% (245/245), done.

remote: Compressing objects:   1% (1/72)
remote: Compressing objects:   2% (2/72)
remote: Compressing objects:   4% (3/72)
remote: Compressing objects:   5% (4/72)
remote: Compressing objects:   6% (5/72)
remote: Compressing objects:   8% (6/72)
remote: Compressing objects:   9% (7/72)
remote: Compressing objects:  11% (8/72)
remote: Compressing objects:  12% (9/72)
remote: Compressing objects:  13% (10/72)
remote: Compressing objects:  15% (11/72)
remote: Compressing objects:  16% (12/72)
remote: Compressing objects:  18% (13/72)
remote: Compressing objects:  19% (14/72)
remote: Compressing objects:  20% (15/72)
remote: Compressing objects:  22% (16/72)
remote: Compressing objects:  23% (17/72)
remote: Compressing objects:  25% (18/72)
remote: Compressing objects:  26% (19/72)
remote: Compressing objects:  27% (20/72)
remote: Compressing objects:  29% (21/72)
remote: Compressing objects:  30% (22/72)
remote: Compressing objects:  31% (23/72)
remote: Compressing objects:  33% (24/72)
remote: Compressing objects:  34% (25/72)
remote: Compressing objects:  36% (26/72)
remote: Compressing objects:  37% (27/72)
remote: Compressing objects:  38% (28/72)
remote: Compressing objects:  40% (29/72)
remote: Compressing objects:  41% (30/72)
remote: Compressing objects:  43% (31/72)
remote: Compressing objects:  44% (32/72)
remote: Compressing objects:  45% (33/72)
remote: Compressing objects:  47% (34/72)
remote: Compressing objects:  48% (35/72)
remote: Compressing objects:  50% (36/72)
remote: Compressing objects:  51% (37/72)
remote: Compressing objects:  52% (38/72)
remote: Compressing objects:  54% (39/72)
remote: Compressing objects:  55% (40/72)
remote: Compressing objects:  56% (41/72)
remote: Compressing objects:  58% (42/72)
remote: Compressing objects:  59% (43/72)
remote: Compressing objects:  61% (44/72)
remote: Compressing objects:  62% (45/72)
remote: Compressing objects:  63% (46/72)
remote: Compressing objects:  65% (47/72)
remote: Compressing objects:  66% (48/72)
remote: Compressing objects:  68% (49/72)
remote: Compressing objects:  69% (50/72)
remote: Compressing objects:  70% (51/72)
remote: Compressing objects:  72% (52/72)
remote: Compressing objects:  73% (53/72)
remote: Compressing objects:  75% (54/72)
remote: Compressing objects:  76% (55/72)
remote: Compressing objects:  77% (56/72)
remote: Compressing objects:  79% (57/72)
remote: Compressing objects:  80% (58/72)
remote: Compressing objects:  81% (59/72)
remote: Compressing objects:  83% (60/72)
remote: Compressing objects:  84% (61/72)
remote: Compressing objects:  86% (62/72)
remote: Compressing objects:  87% (63/72)
remote: Compressing objects:  88% (64/72)
remote: Compressing objects:  90% (65/72)
remote: Compressing objects:  91% (66/72)
remote: Compressing objects:  93% (67/72)
remote: Compressing objects:  94% (68/72)
remote: Compressing objects:  95% (69/72)
remote: Compressing objects:  97% (70/72)
remote: Compressing objects:  98% (71/72)
remote: Compressing objects: 100% (72/72)
remote: Compressing objects: 100% (72/72), done.

Receiving objects:   0% (1/709)
Receiving objects:   1% (8/709)
Receiving objects:   2% (15/709)
Receiving objects:   3% (22/709)
Receiving objects:   4% (29/709)
Receiving objects:   5% (36/709)
Receiving objects:   6% (43/709)
Receiving objects:   7% (50/709)
Receiving objects:   8% (57/709)
Receiving objects:   9% (64/709)
Receiving objects:  10% (71/709)
Receiving objects:  11% (78/709)
Receiving objects:  12% (86/709)
Receiving objects:  13% (93/709)
Receiving objects:  14% (100/709)
Receiving objects:  15% (107/709)
Receiving objects:  16% (114/709)
Receiving objects:  17% (121/709)
Receiving objects:  18% (128/709)
Receiving objects:  19% (135/709)
Receiving objects:  20% (142/709)
Receiving objects:  21% (149/709)
Receiving objects:  22% (156/709)
Receiving objects:  23% (164/709)
Receiving objects:  24% (171/709)
Receiving objects:  25% (178/709)
Receiving objects:  26% (185/709)
Receiving objects:  27% (192/709)
Receiving objects:  28% (199/709)
Receiving objects:  29% (206/709)
Receiving objects:  30% (213/709)
Receiving objects:  31% (220/709)
Receiving objects:  32% (227/709)
Receiving objects:  33% (234/709)
Receiving objects:  34% (242/709)
Receiving objects:  35% (249/709)
Receiving objects:  36% (256/709)
Receiving objects:  37% (263/709)
Receiving objects:  38% (270/709)
Receiving objects:  39% (277/709)
Receiving objects:  40% (284/709)
Receiving objects:  41% (291/709)
Receiving objects:  42% (298/709)
Receiving objects:  43% (305/709)
Receiving objects:  44% (312/709)
Receiving objects:  45% (320/709)
Receiving objects:  46% (327/709)
Receiving objects:  47% (334/709)
Receiving objects:  48% (341/709)
Receiving objects:  49% (348/709)
Receiving objects:  50% (355/709)
Receiving objects:  51% (362/709)
Receiving objects:  52% (369/709)
Receiving objects:  53% (376/709)
Receiving objects:  54% (383/709)
Receiving objects:  55% (390/709)
Receiving objects:  56% (398/709)
Receiving objects:  57% (405/709)
Receiving objects:  58% (412/709)
Receiving objects:  59% (419/709)
Receiving objects:  60% (426/709)
Receiving objects:  61% (433/709)
Receiving objects:  62% (440/709)
Receiving objects:  63% (447/709)
Receiving objects:  64% (454/709)
Receiving objects:  65% (461/709)
Receiving objects:  66% (468/709)
Receiving objects:  67% (476/709)
Receiving objects:  68% (483/709)
Receiving objects:  69% (490/709)
Receiving objects:  70% (497/709)
Receiving objects:  71% (504/709)
remote: Total 709 (delta 185), reused 207 (delta 158), pack-reused 464

Receiving objects:  72% (511/709)
Receiving objects:  73% (518/709)
Receiving objects:  74% (525/709)
Receiving objects:  75% (532/709)
Receiving objects:  76% (539/709)
Receiving objects:  77% (546/709)
Receiving objects:  78% (554/709)
Receiving objects:  79% (561/709)
Receiving objects:  80% (568/709)
Receiving objects:  81% (575/709)
Receiving objects:  82% (582/709)
Receiving objects:  83% (589/709)
Receiving objects:  84% (596/709)
Receiving objects:  85% (603/709)
Receiving objects:  86% (610/709)
Receiving objects:  87% (617/709)
Receiving objects:  88% (624/709)
Receiving objects:  89% (632/709)
Receiving objects:  90% (639/709)
Receiving objects:  91% (646/709)
Receiving objects:  92% (653/709)
Receiving objects:  93% (660/709)
Receiving objects:  94% (667/709)
Receiving objects:  95% (674/709)
Receiving objects:  96% (681/709)
Receiving objects:  97% (688/709)
Receiving objects:  98% (695/709)
Receiving objects:  99% (702/709)
Receiving objects: 100% (709/709)
Receiving objects: 100% (709/709), 118.28 KiB | 2.07 MiB/s, done.

Resolving deltas:   0% (0/342)
Resolving deltas:   1% (4/342)
Resolving deltas:   2% (7/342)
Resolving deltas:   3% (11/342)
Resolving deltas:   4% (14/342)
Resolving deltas:   5% (18/342)
Resolving deltas:   6% (21/342)
Resolving deltas:   7% (24/342)
Resolving deltas:   8% (28/342)
Resolving deltas:   9% (31/342)
Resolving deltas:  10% (35/342)
Resolving deltas:  11% (38/342)
Resolving deltas:  12% (42/342)
Resolving deltas:  13% (45/342)
Resolving deltas:  14% (48/342)
Resolving deltas:  15% (52/342)
Resolving deltas:  16% (55/342)
Resolving deltas:  17% (59/342)
Resolving deltas:  18% (62/342)
Resolving deltas:  19% (65/342)
Resolving deltas:  20% (69/342)
Resolving deltas:  21% (72/342)
Resolving deltas:  22% (76/342)
Resolving deltas:  23% (79/342)
Resolving deltas:  24% (83/342)
Resolving deltas:  25% (86/342)
Resolving deltas:  26% (89/342)
Resolving deltas:  27% (93/342)
Resolving deltas:  28% (96/342)
Resolving deltas:  29% (100/342)
Resolving deltas:  30% (103/342)
Resolving deltas:  31% (107/342)
Resolving deltas:  32% (110/342)
Resolving deltas:  33% (113/342)
Resolving deltas:  34% (117/342)
Resolving deltas:  35% (120/342)
Resolving deltas:  36% (124/342)
Resolving deltas:  37% (127/342)
Resolving deltas:  38% (130/342)
Resolving deltas:  39% (134/342)
Resolving deltas:  40% (137/342)
Resolving deltas:  41% (141/342)
Resolving deltas:  42% (144/342)
Resolving deltas:  43% (148/342)
Resolving deltas:  44% (151/342)
Resolving deltas:  45% (154/342)
Resolving deltas:  46% (158/342)
Resolving deltas:  47% (161/342)
Resolving deltas:  48% (165/342)
Resolving deltas:  49% (168/342)
Resolving deltas:  50% (171/342)
Resolving deltas:  51% (175/342)
Resolving deltas:  52% (178/342)
Resolving deltas:  53% (182/342)
Resolving deltas:  54% (185/342)
Resolving deltas:  55% (189/342)
Resolving deltas:  56% (192/342)
Resolving deltas:  57% (195/342)
Resolving deltas:  58% (199/342)
Resolving deltas:  59% (202/342)
Resolving deltas:  60% (206/342)
Resolving deltas:  61% (209/342)
Resolving deltas:  62% (213/342)
Resolving deltas:  63% (216/342)
Resolving deltas:  64% (219/342)
Resolving deltas:  65% (223/342)
Resolving deltas:  66% (226/342)
Resolving deltas:  67% (230/342)
Resolving deltas:  68% (233/342)
Resolving deltas:  69% (236/342)
Resolving deltas:  70% (240/342)
Resolving deltas:  71% (244/342)
Resolving deltas:  72% (247/342)
Resolving deltas:  73% (250/342)
Resolving deltas:  74% (254/342)
Resolving deltas:  75% (257/342)
Resolving deltas:  76% (261/342)
Resolving deltas:  77% (264/342)
Resolving deltas:  78% (267/342)
Resolving deltas:  79% (271/342)
Resolving deltas:  80% (274/342)
Resolving deltas:  81% (278/342)
Resolving deltas:  82% (281/342)
Resolving deltas:  83% (284/342)
Resolving deltas:  84% (288/342)
Resolving deltas:  85% (291/342)
Resolving deltas:  86% (295/342)
Resolving deltas:  87% (298/342)
Resolving deltas:  88% (301/342)
Resolving deltas:  89% (305/342)
Resolving deltas:  90% (308/342)
Resolving deltas:  91% (312/342)
Resolving deltas:  92% (315/342)
Resolving deltas:  93% (319/342)
Resolving deltas:  94% (322/342)
Resolving deltas:  95% (325/342)
Resolving deltas:  96% (329/342)
Resolving deltas:  97% (332/342)
Resolving deltas:  98% (336/342)
Resolving deltas:  99% (339/342)
Resolving deltas: 100% (342/342)
Resolving deltas: 100% (342/342), done.

# Checking out `1e7f666c6a6752a65c4ae22d28b9150a82af0414`

$ git checkout -f 1e7f666c6a6752a65c4ae22d28b9150a82af0414

Note: switching to '1e7f666c6a6752a65c4ae22d28b9150a82af0414'.

You are in 'detached HEAD' state. You can look around, make experimental

changes and commit them, and you can discard any commits you make in this

state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may

do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 1e7f666 some logs

$ cd /

~~~ Preparing working directory

# Creating "/buildkite/builds/buildkite-agent-6b59766cd4-ft4cj-1/mindful/global-base"

$ cd /buildkite/builds/buildkite-agent-6b59766cd4-ft4cj-1/mindful/global-base

# Added host "github.com" to known hosts at "/root/.ssh/known_hosts"

# Using git-mirrors experiment 🧪

$ cd /git-mirrors

# Cloning a mirror of the repository to "/git-mirrors/git-github-com-mindful-hq-global-base-git"

$ git clone --mirror -v -- git@github.com:mindful-hq/global-base.git /git-mirrors/git-github-com-mindful-hq-global-base-git

Cloning into bare repository '/git-mirrors/git-github-com-mindful-hq-global-base-git'...

remote: Enumerating objects: 307, done.

remote: Counting objects:   2% (1/42)
remote: Counting objects:   4% (2/42)
remote: Counting objects:   7% (3/42)
remote: Counting objects:   9% (4/42)
remote: Counting objects:  11% (5/42)
remote: Counting objects:  14% (6/42)
remote: Counting objects:  16% (7/42)
remote: Counting objects:  19% (8/42)
remote: Counting objects:  21% (9/42)
remote: Counting objects:  23% (10/42)
remote: Counting objects:  26% (11/42)
remote: Counting objects:  28% (12/42)
remote: Counting objects:  30% (13/42)
remote: Counting objects:  33% (14/42)
remote: Counting objects:  35% (15/42)
remote: Counting objects:  38% (16/42)
remote: Counting objects:  40% (17/42)
remote: Counting objects:  42% (18/42)
remote: Counting objects:  45% (19/42)
remote: Counting objects:  47% (20/42)
remote: Counting objects:  50% (21/42)
remote: Counting objects:  52% (22/42)
remote: Counting objects:  54% (23/42)
remote: Counting objects:  57% (24/42)
remote: Counting objects:  59% (25/42)
remote: Counting objects:  61% (26/42)
remote: Counting objects:  64% (27/42)
remote: Counting objects:  66% (28/42)
remote: Counting objects:  69% (29/42)
remote: Counting objects:  71% (30/42)
remote: Counting objects:  73% (31/42)
remote: Counting objects:  76% (32/42)
remote: Counting objects:  78% (33/42)
remote: Counting objects:  80% (34/42)
remote: Counting objects:  83% (35/42)
remote: Counting objects:  85% (36/42)
remote: Counting objects:  88% (37/42)
remote: Counting objects:  90% (38/42)
remote: Counting objects:  92% (39/42)
remote: Counting objects:  95% (40/42)
remote: Counting objects:  97% (41/42)
remote: Counting objects: 100% (42/42)
remote: Counting objects: 100% (42/42), done.

remote: Compressing objects:   4% (1/22)
remote: Compressing objects:   9% (2/22)
remote: Compressing objects:  13% (3/22)
remote: Compressing objects:  18% (4/22)
remote: Compressing objects:  22% (5/22)
remote: Compressing objects:  27% (6/22)
remote: Compressing objects:  31% (7/22)
remote: Compressing objects:  36% (8/22)
remote: Compressing objects:  40% (9/22)
remote: Compressing objects:  45% (10/22)
remote: Compressing objects:  50% (11/22)
remote: Compressing objects:  54% (12/22)
remote: Compressing objects:  59% (13/22)
remote: Compressing objects:  63% (14/22)
remote: Compressing objects:  68% (15/22)
remote: Compressing objects:  72% (16/22)
remote: Compressing objects:  77% (17/22)
remote: Compressing objects:  81% (18/22)
remote: Compressing objects:  86% (19/22)
remote: Compressing objects:  90% (20/22)
remote: Compressing objects:  95% (21/22)
remote: Compressing objects: 100% (22/22)
remote: Compressing objects: 100% (22/22), done.

Receiving objects:   0% (1/307)
Receiving objects:   1% (4/307)
Receiving objects:   2% (7/307)
Receiving objects:   3% (10/307)
Receiving objects:   4% (13/307)
Receiving objects:   5% (16/307)
Receiving objects:   6% (19/307)
Receiving objects:   7% (22/307)
Receiving objects:   8% (25/307)
Receiving objects:   9% (28/307)
Receiving objects:  10% (31/307)
Receiving objects:  11% (34/307)
Receiving objects:  12% (37/307)
Receiving objects:  13% (40/307)
Receiving objects:  14% (43/307)
Receiving objects:  15% (47/307)
Receiving objects:  16% (50/307)
Receiving objects:  17% (53/307)
Receiving objects:  18% (56/307)
Receiving objects:  19% (59/307)
Receiving objects:  20% (62/307)
Receiving objects:  21% (65/307)
Receiving objects:  22% (68/307)
Receiving objects:  23% (71/307)
Receiving objects:  24% (74/307)
Receiving objects:  25% (77/307)
Receiving objects:  26% (80/307)
Receiving objects:  27% (83/307)
Receiving objects:  28% (86/307)
Receiving objects:  29% (90/307)
Receiving objects:  30% (93/307)
Receiving objects:  31% (96/307)
Receiving objects:  32% (99/307)
Receiving objects:  33% (102/307)
Receiving objects:  34% (105/307)
Receiving objects:  35% (108/307)
Receiving objects:  36% (111/307)
Receiving objects:  37% (114/307)
Receiving objects:  38% (117/307)
Receiving objects:  39% (120/307)
Receiving objects:  40% (123/307)
Receiving objects:  41% (126/307)
Receiving objects:  42% (129/307)
Receiving objects:  43% (133/307)
Receiving objects:  44% (136/307)
Receiving objects:  45% (139/307)
Receiving objects:  46% (142/307)
Receiving objects:  47% (145/307)
Receiving objects:  48% (148/307)
Receiving objects:  49% (151/307)
Receiving objects:  50% (154/307)
Receiving objects:  51% (157/307)
Receiving objects:  52% (160/307)
Receiving objects:  53% (163/307)
Receiving objects:  54% (166/307)
Receiving objects:  55% (169/307)
Receiving objects:  56% (172/307)
Receiving objects:  57% (175/307)
Receiving objects:  58% (179/307)
Receiving objects:  59% (182/307)
Receiving objects:  60% (185/307)
Receiving objects:  61% (188/307)
Receiving objects:  62% (191/307)
Receiving objects:  63% (194/307)
Receiving objects:  64% (197/307)
Receiving objects:  65% (200/307)
Receiving objects:  66% (203/307)
Receiving objects:  67% (206/307)
Receiving objects:  68% (209/307)
Receiving objects:  69% (212/307)
Receiving objects:  70% (215/307)
remote: Total 307 (delta 21), reused 24 (delta 6), pack-reused 265

Receiving objects:  71% (218/307)
Receiving objects:  72% (222/307)
Receiving objects:  73% (225/307)
Receiving objects:  74% (228/307)
Receiving objects:  75% (231/307)
Receiving objects:  76% (234/307)
Receiving objects:  77% (237/307)
Receiving objects:  78% (240/307)
Receiving objects:  79% (243/307)
Receiving objects:  80% (246/307)
Receiving objects:  81% (249/307)
Receiving objects:  82% (252/307)
Receiving objects:  83% (255/307)
Receiving objects:  84% (258/307)
Receiving objects:  85% (261/307)
Receiving objects:  86% (265/307)
Receiving objects:  87% (268/307)
Receiving objects:  88% (271/307)
Receiving objects:  89% (274/307)
Receiving objects:  90% (277/307)
Receiving objects:  91% (280/307)
Receiving objects:  92% (283/307)
Receiving objects:  93% (286/307)
Receiving objects:  94% (289/307)
Receiving objects:  95% (292/307)
Receiving objects:  96% (295/307)
Receiving objects:  97% (298/307)
Receiving objects:  98% (301/307)
Receiving objects:  99% (304/307)
Receiving objects: 100% (307/307)
Receiving objects: 100% (307/307), 59.27 KiB | 2.19 MiB/s, done.

Resolving deltas:   0% (0/163)
Resolving deltas:   1% (2/163)
Resolving deltas:   2% (4/163)
Resolving deltas:   3% (5/163)
Resolving deltas:   4% (7/163)
Resolving deltas:   5% (9/163)
Resolving deltas:   6% (10/163)
Resolving deltas:   7% (12/163)
Resolving deltas:   8% (14/163)
Resolving deltas:   9% (15/163)
Resolving deltas:  10% (17/163)
Resolving deltas:  11% (18/163)
Resolving deltas:  12% (20/163)
Resolving deltas:  13% (22/163)
Resolving deltas:  14% (23/163)
Resolving deltas:  15% (25/163)
Resolving deltas:  16% (27/163)
Resolving deltas:  17% (28/163)
Resolving deltas:  18% (30/163)
Resolving deltas:  19% (31/163)
Resolving deltas:  20% (33/163)
Resolving deltas:  21% (35/163)
Resolving deltas:  22% (36/163)
Resolving deltas:  23% (38/163)
Resolving deltas:  24% (40/163)
Resolving deltas:  25% (41/163)
Resolving deltas:  26% (43/163)
Resolving deltas:  27% (45/163)
Resolving deltas:  28% (46/163)
Resolving deltas:  29% (48/163)
Resolving deltas:  30% (49/163)
Resolving deltas:  31% (51/163)
Resolving deltas:  32% (53/163)
Resolving deltas:  33% (54/163)
Resolving deltas:  34% (56/163)
Resolving deltas:  35% (58/163)
Resolving deltas:  36% (59/163)
Resolving deltas:  37% (61/163)
Resolving deltas:  38% (62/163)
Resolving deltas:  39% (64/163)
Resolving deltas:  40% (66/163)
Resolving deltas:  41% (67/163)
Resolving deltas:  42% (69/163)
Resolving deltas:  43% (71/163)
Resolving deltas:  44% (72/163)
Resolving deltas:  45% (74/163)
Resolving deltas:  46% (75/163)
Resolving deltas:  47% (77/163)
Resolving deltas:  48% (79/163)
Resolving deltas:  49% (80/163)
Resolving deltas:  50% (82/163)
Resolving deltas:  51% (84/163)
Resolving deltas:  52% (85/163)
Resolving deltas:  53% (87/163)
Resolving deltas:  54% (89/163)
Resolving deltas:  55% (90/163)
Resolving deltas:  56% (92/163)
Resolving deltas:  57% (93/163)
Resolving deltas:  58% (95/163)
Resolving deltas:  59% (97/163)
Resolving deltas:  60% (98/163)
Resolving deltas:  61% (100/163)
Resolving deltas:  62% (102/163)
Resolving deltas:  63% (103/163)
Resolving deltas:  64% (105/163)
Resolving deltas:  65% (106/163)
Resolving deltas:  66% (108/163)
Resolving deltas:  67% (110/163)
Resolving deltas:  68% (111/163)
Resolving deltas:  69% (113/163)
Resolving deltas:  70% (115/163)
Resolving deltas:  71% (116/163)
Resolving deltas:  72% (118/163)
Resolving deltas:  73% (119/163)
Resolving deltas:  74% (121/163)
Resolving deltas:  75% (123/163)
Resolving deltas:  76% (124/163)
Resolving deltas:  77% (126/163)
Resolving deltas:  78% (128/163)
Resolving deltas:  79% (129/163)
Resolving deltas:  80% (131/163)
Resolving deltas:  81% (133/163)
Resolving deltas:  82% (134/163)
Resolving deltas:  83% (136/163)
Resolving deltas:  84% (137/163)
Resolving deltas:  85% (139/163)
Resolving deltas:  86% (141/163)
Resolving deltas:  87% (142/163)
Resolving deltas:  88% (144/163)
Resolving deltas:  89% (146/163)
Resolving deltas:  90% (147/163)
Resolving deltas:  91% (149/163)
Resolving deltas:  92% (150/163)
Resolving deltas:  93% (152/163)
Resolving deltas:  94% (154/163)
Resolving deltas:  95% (155/163)
Resolving deltas:  96% (157/163)
Resolving deltas:  97% (159/163)
Resolving deltas:  98% (160/163)
Resolving deltas:  99% (162/163)
Resolving deltas: 100% (163/163)
Resolving deltas: 100% (163/163), done.

$ cd /buildkite/builds/buildkite-agent-6b59766cd4-ft4cj-1/mindful/global-base

$ git clone -v --reference /git-mirrors/git-github-com-mindful-hq-global-base-git -- git@github.com:mindful-hq/global-base.git .

Cloning into '.'...

$ git clean -ffxdq

$ git fetch -v --prune -- origin d7ed53a809c877cab0d827dfbfc0caea49613a26

From github.com:mindful-hq/global-base

 * branch            d7ed53a809c877cab0d827dfbfc0caea49613a26 -> FETCH_HEAD

$ git checkout -f d7ed53a809c877cab0d827dfbfc0caea49613a26

Note: switching to 'd7ed53a809c877cab0d827dfbfc0caea49613a26'.

You are in 'detached HEAD' state. You can look around, make experimental

changes and commit them, and you can discard any commits you make in this

state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may

do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at d7ed53a test: buildkite

# Cleaning again to catch any post-checkout changes

$ git clean -ffxdq

# Checking to see if Git data needs to be sent to Buildkite

$ buildkite-agent meta-data exists buildkite:git:commit

~~~ Running plugin k8s command hook

$ /buildkite/plugins/github-com-EmbarkStudios-k8s-buildkite-plugin-1e7f666c6a6752a65c4ae22d28b9150a82af0414/hooks/command

$ cd /buildkite/builds/buildkite-agent-6b59766cd4-ft4cj-1/mindful/global-base

~~~ Running plugin k8s pre-exit hook

$ /buildkite/plugins/github-com-EmbarkStudios-k8s-buildkite-plugin-1e7f666c6a6752a65c4ae22d28b9150a82af0414/hooks/pre-exit

$ cd /buildkite/builds/buildkite-agent-6b59766cd4-ft4cj-1/mindful/global-base

Finished tailing logs...

+++ :kubernetes: step container

Polling for pod to start

Starting to tail logs...

Extracting Bazel installation...

Starting local Bazel server and connecting to it...

Loading: 

Loading: 0 packages loaded

Loading: 3 packages loaded

    currently loading: 

Loading: 3 packages loaded

    currently loading: 

Loading: 3 packages loaded

    currently loading: 

Loading: 3 packages loaded

    currently loading: 

Loading: 3 packages loaded

    currently loading: 

Loading: 3 packages loaded

    currently loading: 

Loading: 3 packages loaded

    currently loading: 

Loading: 3 packages loaded

    currently loading: 

Loading: 3 packages loaded

    currently loading: 

Analyzing: 11 targets (4 packages loaded, 0 targets configured)

Analyzing: 11 targets (9 packages loaded, 26 targets configured)

Analyzing: 11 targets (9 packages loaded, 26 targets configured)

Analyzing: 11 targets (9 packages loaded, 26 targets configured)

Analyzing: 11 targets (9 packages loaded, 37 targets configured)

Analyzing: 11 targets (17 packages loaded, 61 targets configured)

Analyzing: 11 targets (45 packages loaded, 255 targets configured)

Analyzing: 11 targets (45 packages loaded, 255 targets configured)

Analyzing: 11 targets (45 packages loaded, 255 targets configured)

Analyzing: 11 targets (45 packages loaded, 255 targets configured)

Analyzing: 11 targets (45 packages loaded, 255 targets configured)

Analyzing: 11 targets (45 packages loaded, 255 targets configured)

Analyzing: 11 targets (45 packages loaded, 255 targets configured)

Analyzing: 11 targets (45 packages loaded, 255 targets configured)

Analyzing: 11 targets (50 packages loaded, 399 targets configured)

Analyzing: 11 targets (50 packages loaded, 402 targets configured)

Analyzing: 11 targets (50 packages loaded, 402 targets configured)

Analyzing: 11 targets (50 packages loaded, 402 targets configured)

Analyzing: 11 targets (56 packages loaded, 466 targets configured)

Analyzing: 11 targets (56 packages loaded, 471 targets configured)

INFO: Analyzed 11 targets (57 packages loaded, 475 targets configured).

 checking cached actions

INFO: Found 9 targets and 2 test targets...

WARNING: failed to create one or more convenience symlinks for prefix 'bazel-':

  cannot create symbolic link bazel-bin -> /home/ubuntu/.cache/bazel/_bazel_ubuntu/7b7747ec045ae606eb720a1222f56098/execroot/__main__/bazel-out/k8-fastbuild/bin:  /build/bazel-bin (Permission denied)

  cannot create symbolic link bazel-testlogs -> /home/ubuntu/.cache/bazel/_bazel_ubuntu/7b7747ec045ae606eb720a1222f56098/execroot/__main__/bazel-out/k8-fastbuild/testlogs:  /build/bazel-testlogs (Permission denied)

  cannot create symbolic link bazel-out -> /home/ubuntu/.cache/bazel/_bazel_ubuntu/7b7747ec045ae606eb720a1222f56098/execroot/__main__/bazel-out:  /build/bazel-out (Permission denied)

  cannot create symbolic link bazel-build -> /home/ubuntu/.cache/bazel/_bazel_ubuntu/7b7747ec045ae606eb720a1222f56098/execroot/__main__:  /build/bazel-build (Permission denied)

[0 / 1] [Prepa] BazelWorkspaceStatusAction stable-status.txt

[11 / 12] 1 / 2 tests; Testing //deployments/production:tflint; 1s processwrapper-sandbox

INFO: Elapsed time: 134.766s, Critical Path: 2.05s

INFO: 12 processes: 8 internal, 4 processwrapper-sandbox.

INFO: Build completed successfully, 12 total actions

//:ls-lint                                                               PASSED in 0.1s

//deployments/production:tflint                                          PASSED in 1.9s

Executed 2 out of 2 tests: 2 tests pass.

There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

Finished tailing logs...

--- :kubernetes: Job status: Complete

success

~~~ Running plugin k8s pre-exit hook
$ /buildkite/plugins/github-com-EmbarkStudios-k8s-buildkite-plugin-1e7f666c6a6752a65c4ae22d28b9150a82af0414/hooks/pre-exit
--- :kubernetes: Cleanup
loeffel-io commented 1 year ago

@tgolsson how can i access the ssh key from inside the bazel container, is there an easy way? thanks!

tgolsson commented 1 year ago

You'll need to look at the environment-from-secret or mount-secret options for that :-)

loeffel-io commented 1 year ago

could you please give me an example on how to do that with the existing key?

mount-secret: "git-ssh-key:/secrets/ssh-key" produces 

The Job "global-base-104-kmacfl6c" is invalid: spec.template.spec.volumes[4].name: Duplicate value: "git-ssh-key"
--
  | Attempt to apply the job failed, exit code '1'

mount-secret: "agent-ssh:/secrets/ssh-key" makes the container stuck

tgolsson commented 1 year ago

I think you need non-overlapping names and paths. I don't know enough about your system and config, if agent-ssh is a valid secret you might be able to mount it as agent-ssh:/agent-ssh - that should put all keys from that inside the /agent-ssh folder. The reason it gets stuck is that on the init container you'll have two sources for /secrets/ssh-key - once from the default/setup keys and once from the extra mount-secret.

(If things get stuck; I'd look at the kubernetes events for the jobs - it's likely failing to start there.)

loeffel-io commented 1 year ago

all i want is to reuse my already existing secret from the plugin

secret-name: "buildkite-agent"
agent-token-secret-key: "agent-token"
git-ssh-secret-key: "agent-ssh"
mount-secret: ?

how? 😄

loeffel-io commented 1 year ago

another issue i face is that when i want to create my own buildkite init image the bootstrap command is not called with the same entrypoint script

custom

exec /usr/local/bin/buildkite-agent

embarkstudios/k8s-buildkite-agent@sha256:1d88791315ed6b0b49a64055bc71c5a9a0b1953e387f99d25299ed06ccea5dbd

exec /usr/local/bin/buildkite-agent bootstrap --experiment=git-mirrors --git-mirrors-path=/git-mirrors --ssh-keyscan --command true

at this line: exec /usr/local/bin/buildkite-agent "$@"

tgolsson commented 1 year ago

all i want is to reuse my already existing secret from the plugin

I'll think about how we can simplify this, but agent-ssh:/some-dir should work. I wasn't maintainer when it was setup so I'm not sure if having different sets of secrets is intentional or an oversight. Would need to dig in the history a bit.

another issue i face is that when i want to create my own buildkite init image the bootstrap command is not called with the same entrypoint script

That sounds weird. The command line is hard-coded when we generate the job, so it sounds like the entrypoint isn't forwarding them properly.

loeffel-io commented 1 year ago

I'll think about how we can simplify this, but agent-ssh:/some-dir should work. I wasn't maintainer when it was setup so I'm not sure if having different sets of secrets is intentional or an oversight. Would need to dig in the history a bit.

so i need to create a new secret with the exact same keys and values right? i think a solution could be to just check if the mount-secret secret already exists in the list at plugin level and then stop adding it to the list - problem then is that it must be the same path ..

tgolsson commented 1 year ago

so i need to create a new secret with the exact same keys and values right.

Yes. The only secret that is shared between init and main pod is the mount-secret config, and that has to use a name that isn't also used elsewhere. We could probably dedupe it, but the secrets already seem far too complicated, so I'd rather go in the direction of just mounting everything everywhere. There is I think 3 ways of configuring SSH-key for example... That seems excessive.

loeffel-io commented 1 year ago

thanks again @tgolsson 🙏 everything is working now with some workarounds would love to see some bug fixes to this project thank you so much ❤️