firecow / gitlab-ci-local

Tired of pushing to test your .gitlab-ci.yml?
MIT License
2.03k stars 115 forks source link

How can I get the external path when running gitlab ci-local jobs #1230

Closed WCY91 closed 1 month ago

WCY91 commented 1 month ago

Minimal .gitlab-ci.yml illustrating the issue

---
job:
  script:
     - echo "$PRIVATE_KEY" > private_key.key
    - gpg --import private_key.key

Expected behavior it should get the key which is on my pc

Host information Mac gitlab-ci-local latest

Containerd binary Are you using docker or podman, or any other relevant containerization tool.

Additional context the private key path is like : "/User//" and print the path by echo cmd is correct value

PigeonF commented 1 month ago

I am not sure what the question is, so I am going to answer how I understood your question: how to map a file from the host system into the job run by gitlab-ci-local.

Using a Shell executor

Using the shell executor with the sample above seems to work fine (you can tell it uses the shell executor, as there is no image:), since gitlab-ci-local has access to your host filesystem.

---
job:
  script:
    - echo "$PRIVATE_KEY"
    - ls -la $PRIVATE_KEY
    - cat $PRIVATE_KEY
$ echo "I am the private key" > private.key
$ gitlab-ci-local --variable PRIVATE_KEY="$(pwd)/private.key"
job starting shell (test)
job $ echo "$PRIVATE_KEY"
job > /tmp/tmp.KnyhyOlxKI/private.key
job $ ls -la $PRIVATE_KEY
job > -rw-r--r-- 1 pigeon pigeon 21 22. Mai 11:11 /tmp/tmp.KnyhyOlxKI/private.key
job $ cat $PRIVATE_KEY
job > I am the private key
job finished in 28 ms

 PASS  job
pipeline finished in 142 ms

Using a docker executor

If you are using the docker executor, you can map the private key as a volume, though to reproduce this on GitLab, you would have to have access to the gitlab runner configuration file

---
job:
  image: docker.io/busybox
  script:
    - echo "$PRIVATE_KEY"
    - ls -la $PRIVATE_KEY
    - cat $PRIVATE_KEY
$ echo "I am the private key" > private.key
$ gitlab-ci-local --variable PRIVATE_KEY="/secrets/private.key" --volume "$(pwd)/private.key:/secrets/private.key:ro"
job starting docker.io/busybox:latest (test)
job copied to docker volumes in 572 ms
job $ echo "$PRIVATE_KEY"
job > /secrets/private.key
job $ ls -la $PRIVATE_KEY
job > -rw-r--r--    1 1000     996             21 May 22 09:11 /secrets/private.key
job $ cat $PRIVATE_KEY
job > I am the private key
job finished in 1.5 s

 PASS  job
pipeline finished in 1.62 s
PigeonF commented 1 month ago
$ cd $(mktemp -d)
$ mkdir gnupg && chmod 0700 gnupg
$ export GNUPGHOME="$(pwd)/gnupg"
$ cat > key.batch <<EOF
%no-protection
Key-Type: DSA
Key-Length: 1024
Subkey-Type: ELG-E
Subkey-Length: 1024
Name-Real: John Doe
Expire-Date: 0
EOF
$ gpg --generate-key --batch key.batch
$ gpg --armor --export-secret-keys "John Doe" > private_key.key
$ echo "Hello, World" | gpg -e -r "John Doe" --armor > message.gpg.asc

With the following .gitlab-ci.yml

---
before_script:
  - export GNUPGHOME="$(mktemp -d)"
  - gpg --list-keys
  - ls -l private_key.key
  #- cat private_key.key
  - gpg --import private_key.key
  - gpg --list-keys

job:
  script:
    - ls -l message.gpg.asc
    #- cat message.gpg.asc
    - gpg --decrypt message.gpg.asc
$ gitlab-ci-local
job starting shell (test)
job $ export GNUPGHOME="$(mktemp -d)"
job $ gpg --list-keys
job > gpg: keybox '/run/user/1000/tmp.q8dppqKNvt/pubring.kbx' created
job > gpg: /run/user/1000/tmp.q8dppqKNvt/trustdb.gpg: trustdb created
job $ ls -l private_key.key
job > -rw-r--r-- 1 pigeon pigeon 1422 May 22 11:37 private_key.key
job $ gpg --import private_key.key
job > gpg: key 24CDC076A0D51FED: public key "John Doe" imported
job > gpg: key 24CDC076A0D51FED: secret key imported
job > gpg: Total number processed: 1
job > gpg:               imported: 1
job > gpg:       secret keys read: 1
job > gpg:   secret keys imported: 1
job $ gpg --list-keys
job > /run/user/1000/tmp.q8dppqKNvt/pubring.kbx
job > -----------------------------------------
job > pub   dsa1024 2024-05-22 [SCA]
job >       E299E68E830E08529F3D62EF24CDC076A0D51FED
job > uid           [ unknown] John Doe
job > sub   elg1024 2024-05-22 [E]
job >
job $ ls -l message.gpg.asc
job > -rw-r--r-- 1 pigeon pigeon 545 May 22 11:39 message.gpg.asc
job $ gpg --decrypt message.gpg.asc
job > gpg: encrypted with elg1024 key, ID 1B8968D1F8B80D75, created 2024-05-22
job >       "John Doe"
job > Hello, World
job finished in 70 ms

 PASS  job
pipeline finished in 191 ms

I am using gpg 2.4.5.

Is PRIVATE_KEY set to a path or the actual private key content? It seems to me that the issue lies in your .gitlab-ci.yml or the way you set PRIVATE_KEY.

PigeonF commented 1 month ago

You set "$PRIVATE_KEY_PATH" to a path, but then write it to a file, as though it were the actual key. Instead, you should try

before_script:
  - gpg --import "$PRIVATE_KEY_PATH"

job:
  stage: stage
  script:
  - gpg --decrypt config.py.gpg > config.py
  - pipenv run python3 attendance_check.py
WCY91 commented 1 month ago

thanks for reply it can work ~

ANGkeith commented 1 month ago

thanks for reply it can work ~