search

firecow / gitlab-ci-local

Tired of pushing to test your .gitlab-ci.yml?
MIT License
2.05k stars 118 forks source link

build directory not found #1148

Closed thoroc closed 4 months ago

thoroc commented 4 months ago

I have tried to run gitlab-ci-local without changes, and it seems to be failing on copying the files to the containers.

gitlab-ci-local
parsing and downloads finished in 282 ms
dependencies          starting <aws_account_id>.dkr.ecr.eu-central-1.amazonaws.com/alpine-nodejs:18.19.0 (.pre)
dependencies          copied to docker volumes in 4.16 s
Error: ENOENT: no such file or directory, stat '/Users/thoroc/work/project//gcl-builds/package-lock.json'

I can see a .gitlab-ci-local directory that seems to be the one gitlab-ci-local is looking for as there is a copy of the current dir inside a build sub directory. It also contains an output directory, a .gitignore file and a state.yml file.

firecow commented 4 months ago

What .gitlab-ci-yml are you using? What version of gitlab-ci-local are you using?

thoroc commented 4 months ago

The gitlab-ci.yml in the work project is quite hefty, but I have tried with the following to reproduce it. Mind that I have a bare bone package.json with a single dependency on cowsay (https://www.npmjs.com/package/cowsay) for the sake of introducing a node dependency

default:
  image: ${AWS_ECR}/alpine-nodejs:18.19.0

workflow:
  auto_cancel:
    on_new_commit: interruptible

.cache_node: &cache_node
  key:
    files:
      - ${CI_PROJECT_DIR}/package-lock.json
  paths:
    - ${CI_PROJECT_DIR}/node_modules

stages:
  - say

checks:
  stage: .pre
  # cache:
  #   - <<: *cache_node
  #     policy: pull-push
  script: |
    echo ${CI_PROJECT_DIR}
    ls ${CI_PROJECT_DIR}
    npm config set audit false
    npm config set cache ${CI_PROJECT_DIR}/.npm
    npm ci

say_hello:
  stage: say
  script:
    - echo "Heya"

say_cow:
  stage: say
  script:
    - cowsay "Moo"

say_goodbye:
  stage: say
  script:
    - echo "Goodbye"

The output is not what I had expected:

gitlab-ci-local
git remote -v didn't provide valid matches
Using fallback branch data
parsing and downloads finished in 104 ms
checks      starting <aws_account_id>.dkr.ecr.eu-central-1.amazonaws.com/alpine-nodejs:18.19.0 (.pre)
checks      copied to docker volumes in 695 ms
checks      $ echo ${CI_PROJECT_DIR} # collapsed multi-line command
checks      > /gcl-builds
checks      finished in 2.11 s
say_hello   starting <aws_account_id>.dkr.ecr.eu-central-1.amazonaws.com/alpine-nodejs:18.19.0 (say)
say_cow     starting <aws_account_id>.dkr.ecr.eu-central-1.amazonaws.com/alpine-nodejs:18.19.0 (say)
say_goodbye starting <aws_account_id>.dkr.ecr.eu-central-1.amazonaws.com/alpine-nodejs:18.19.0 (say)
say_hello   copied to docker volumes in 908 ms
say_cow     copied to docker volumes in 1.09 s
say_goodbye copied to docker volumes in 1.18 s
say_hello   $ echo "Heya"
say_hello   > Heya
say_cow     $ cowsay "Moo"
say_cow     > /gcl-cmd: line 4: cowsay: command not found
say_hello   finished in 2.08 s
say_cow     finished in 2.18 s  FAIL 127
say_goodbye $ echo "Goodbye"
say_goodbye > Goodbye
say_goodbye finished in 2.38 s

 PASS  checks
 PASS  say_hello
 PASS  say_goodbye
 FAIL  say_cow
  > /gcl-cmd: line 4: cowsay: command not found
pipeline finished in 4.69 s

Is the variable CI_PROJECT_DIR something I need to pass to gitlab-ci-local? I didn't see anything in the documentation regarding that.

ANGkeith commented 4 months ago

Thanks for the bug report.

See if this fixes your issue.

    files:
      - ${CI_PROJECT_DIR}/package-lock.json    =>  - package-lock.json
  paths:
    - ${CI_PROJECT_DIR}/node_modules           =>   - node_modules
thoroc commented 4 months ago

Removing ${CI_PROJECT_DIR} from those paths definitely removes the issue with gitlab-ci-local. Is that something that is expected?

Just checked and in any of the jobs I have tested (in with the script part) $CI_PROJECT_DIR seems to be the same as $PWD: /gcl-builds

$ gitlab-ci-local
git remote -v didn't provide valid matches
Using fallback branch data
parsing and downloads finished in 161 ms
checks      starting <aws_account_id>.dkr.ecr.eu-central-1.amazonaws.com/alpine-nodejs:18.19.0 (.pre)
checks      copied to docker volumes in 975 ms
checks      $ echo "Current project dir: ${CI_PROJECT_DIR}" # collapsed multi-line command
checks      > Current project dir: /gcl-builds
checks      > total 40K
checks      > drwxrwxrwx    3 root     root        4.0K Mar 18 10:32 .
checks      > drwxr-xr-x    1 root     root        4.0K Mar 18 10:32 ..
checks      > drwxrwxrwx    8 root     root        4.0K Mar 15 14:34 .git
checks      > -rw-rw-rw-    1 root     root          14 Mar 15 14:33 .gitignore
checks      > -rw-rw-rw-    1 root     root         700 Mar 18 10:28 .gitlab-ci.yml
checks      > -rw-rw-rw-    1 root     root       15.4K Mar 15 14:19 package-lock.json
checks      > -rw-rw-rw-    1 root     root         258 Mar 15 14:19 package.json
checks      > Current directory: /gcl-builds
checks      > total 40K
checks      > drwxrwxrwx    3 root     root        4.0K Mar 18 10:32 .
checks      > drwxr-xr-x    1 root     root        4.0K Mar 18 10:32 ..
checks      > drwxrwxrwx    8 root     root        4.0K Mar 15 14:34 .git
checks      > -rw-rw-rw-    1 root     root          14 Mar 15 14:33 .gitignore
checks      > -rw-rw-rw-    1 root     root         700 Mar 18 10:28 .gitlab-ci.yml
checks      > -rw-rw-rw-    1 root     root       15.4K Mar 15 14:19 package-lock.json
checks      > -rw-rw-rw-    1 root     root         258 Mar 15 14:19 package.json
checks      >
checks      > added 41 packages in 2s
checks      >
checks      > 3 packages are looking for funding
checks      >   run `npm fund` for details
checks      finished in 6.4 s
say_hello   starting <aws_account_id>.dkr.ecr.eu-central-1.amazonaws.com/alpine-nodejs:18.19.0 (say)
say_cow     starting <aws_account_id>.dkr.ecr.eu-central-1.amazonaws.com/alpine-nodejs:18.19.0 (say)
say_goodbye starting <aws_account_id>.dkr.ecr.eu-central-1.amazonaws.com/alpine-nodejs:18.19.0 (say)
say_cow     copied to docker volumes in 1.13 s
say_hello   copied to docker volumes in 1.23 s
say_goodbye copied to docker volumes in 1.21 s
say_cow     $ npx cowsay "Moo"
say_goodbye $ echo "Goodbye"
say_goodbye > Goodbye
say_hello   $ echo "Heya"
say_hello   > Heya
say_goodbye finished in 2.61 s
say_hello   finished in 2.69 s
say_cow     > npm WARN exec The following package was not found and will be installed: cowsay@1.6.0
say_cow     >  _____
say_cow     > < Moo >
say_cow     >  -----
say_cow     >         \   ^__^
say_cow     >          \  (oo)\_______
say_cow     >             (__)\       )\/\
say_cow     >                 ||----w |
say_cow     >                 ||     ||
say_cow     finished in 11 s
ANGkeith commented 4 months ago

Is that something that is expected?

nope.

problem is that when we're calculating the hash of the cache key, they're done in the host environment not the container environment.

specifying ${CI_PROJECT_DIR}/package-lock.json references the absolute path of the container environment, but we're expecting a relative path from cwd