Playbook not found when running in gitlab pipeline

[tom0010]

ISSUE TYPE

• Bug Report

SUMMARY

I'm trying to run ansible-navigator to test my execution environment that I built in a gitlab pipeline. I have some simple tasks inside my test playbook, which runs fine on my local machine, however it fails to find the playbook when inside the gitlab pipeline. Bit more detail here: https://ansiblenetwork.slack.com/archives/C80BH7W1E/p1653475017915959

ANSIBLE-NAVIGATOR VERSION

ansible-navigator 2.1.0

CONFIGURATION

None, default.

LOG FILE

https://gist.github.com/tom0010/b28392a681bc10dd6e1e18e865812dc6

STEPS TO REPRODUCE

Use my Dockerfile:

FROM quay.io/ansible/awx-ee:latest

USER root
COPY <my_corp_certs>
RUN update-ca-trust extract
RUN pip install -r requirements.txt && \
    ansible-galaxy collection install -r requrirements.yml -p /usr/share/ansible/collections
USER 1000

Pipeline:

---
stages:
  - "test"

test:
  image: "debian:latest"
  stage: "test"
  before_script:
    - "set -vx"
    - "apt-get update"
    - "apt-get install -y curl gnupg lsb-release"
    - "curl -fsSL https://download.docker.com/linux/debian/gpg |
      gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg"
    - "echo \"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg]
      https://download.docker.com/linux/debian $(lsb_release -cs) stable \"
      | tee /etc/apt/sources.list.d/docker.list > /dev/null"
    - "apt-get update && apt-get install -y docker-ce docker-ce-cli containerd.io"
    - "docker run hello-world"
    - "docker build . -t awx-execution-env:0.2"
  script:
    - "apt-get update"
    - "apt-get install -y python3-pip python3-wheel gcc python3-dev musl-dev libffi-dev"
    - "pip install ansible-navigator[ansible-core]"
    - "ls -la ${CI_PROJECT_DIR}"
    - "ls -la tests/main.yml"
      #    - "ansible-playbook tests/main.yml -vvvv"  <<< it finds this and runs
    - "cat tests/main.yml"
    - "ansible-navigator run --eei
      awx-execution-env:0.2 tests/main.yml --mode stdout -vvvv --ll debug --la false"

EXPECTED RESULTS

For ansible navigator to run the tests/main.yml playbook.

ACTUAL RESULTS

ansible-navigator run --eei awx-execution-env:0.2 tests/main.yml --mode stdout -vvv
ansible-playbook [core 2.12.5.post0]
  config file = None
  configured module search path = ['/home/runner/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/site-packages/ansible
  ansible collection location = /home/runner/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible-playbook
  python version = 3.8.12 (default, Sep 21 2021, 00:10:52) [GCC 8.5.0 20210514 (Red Hat 8.5.0-3)]
  jinja version = 3.1.2
  libyaml = True
No config file found; using defaults
ERROR! the playbook: /builds/netauto/ansible/awx-ee/tests/main.yml could not be found
Please review the log for errors.

ADDITIONAL INFORMATION

Not sure if it is related to https://github.com/ansible/ansible-navigator/issues/1231

[max06]

You might want to switch to docker-in-docker-images, if it doesn't solve your issue, it makes it way easier at least.

[tom0010]

@max06 I actually tried that before using Debian, but that is based on alpine in which onigurumacffi does not play well. I couldn’t get navigator to install on an alpine based image because of that package.

[max06]

I thought about using dind to run your debian image, so you don't have to provide your own docker engine. Basically image docker, service docker:dind and then do a docker run/build/whatever with your preferred image. (It's late here, I might went wrong somewhere)

[cidrblock]

Can you provide a full debug log? ( --ll debug --la false) This will show what got mounted

[tom0010]

@cidrblock that’s in the GitHub gist link in the OP.

[cidrblock]

TY, completely missed that :)

[cidrblock]

extracted from the log:

docker run --rm --tty --interactive 
-v /builds/netauto/ansible/awx-ee/:/builds/netauto/ansible/awx-ee/ 
--workdir /builds/netauto/ansible/awx-ee 
-v /builds/netauto/ansible/awx-ee/tests/:/builds/netauto/ansible/awx-ee/tests/ 
-v /tmp/ansible-navigator_4nhh7skx/artifacts/:/runner/artifacts/:Z 
-v /tmp/ansible-navigator_4nhh7skx/:/runner/:Z 
-v /usr/local/lib/python3.9/dist-packages/ansible_runner/display_callback/callback/:/home/runner/.ansible/plugins/callback/:Z 
--env-file /tmp/ansible-navigator_4nhh7skx/artifacts/1de67569-276b-453d-84d7-703b6e2687c2/env.list 
--user=0 
--name ansible_runner_1de67569-276b-453d-84d7-703b6e2687c2 
awx-execution-env:0.2 
ansible-playbook /builds/netauto/ansible/awx-ee/tests/main.yml -vvvv

CRITICAL 'ansible_navigator.actions.run_playbook._handle_message' Unhandled message from runner queue, discarded: 
{'event': 'verbose', 'uuid': 'a639ce26-50f6-4dac-96dc-122983f975ae', 'counter': 13, 'stdout': 
'\x1b[0;31mERROR! the playbook: /builds/netauto/ansible/awx-ee/tests/main.yml could not be found\x1b[0m', 
'start_line': 12, 'end_line': 13, 'runner_ident': '1de67569-276b-453d-84d7-703b6e2687c2', 'created': '2022-05-26T08:36:53.888923'}

The only thing that concerns me is the volume mounts, I wonder if the first is some how effecting the second, both shouldn't be necessary.... will test here in the next day or so.

In the meantime, can you try ansible-navigator exec --ll debug --la false you should get a shell in the EE, check if the directory and playbook are there, maybe also look for the docker command in the log, I'm guessing it won't have the duplicate (child) mount because no playbook was specified...

[tom0010]

hey @cidrblock actually I'm not getting a shell when using that command when using the gitlab runner, maybe that is related?

$ TERM=xterm256color ansible-navigator exec -i --ll debug --la false --eei awx-execution-env:0.2'
the input device is not a TTY

But the file seems to be there when I run it locally:

poetry run ansible-navigator exec -i --ll debug --la false --eei awx-execution-env:0.2
bash-4.4$ ls -la tests/main.yml
-rw-r--r-- 1 runner 1647400513 1769 May 25 16:30 tests/main.yml

[tom0010]

@cidrblock any news on this please?

[shade-belisar]

I'm running into what I presume to be the same issue. I'm using an image based on this Dockerfile:

FROM docker:26.0.1
RUN apk update
RUN apk add python3 python3-dev py3-pip gcc libc-dev oniguruma-dev
ENV VIRTUAL_ENV=/opt/venv
RUN python3 -m venv $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
RUN pip3 install ansible-navigator

This is then used to run this gitlab ci job:

deploy-to-live:
  stage: deploy
  image: <custom image, see above>
  services:
    - docker:26.0.1-dind
  script:
    - cat /builds/<company-specific path>/00_ubuntu_hardening.yml
    - >-
      ansible-navigator run
      00_ubuntu_hardening.yml
      -i inventories/${INVENTORY}

This results in the following error:

ERROR! the playbook: /builds/<company-specific path>/00_ubuntu_hardening.yml could not be found

However, the cat command did show that the file exists and had the expected content.