[BUG] Failed to generate hosts.yaml

[Woljtek]

Environment: Platform: OPS Orange Cloud Configuration: OPS Bastion: Ubuntu 20.04.5 LTS (Focal Fossa) RS config : ops v1.4

Traçability: Deployment

Current Behavior: I deployed a new bastion from a Ubuntu OS. I follow the deployment quickstart procedure from the infrastructure repository including in RS-config. At step 5, at got the following error:

coprs @ COPRS-OPS-1 ~/Documents/2-OPS/rs-config/infrastructure  ((HEAD detached at 5c41537))
└─ $ ▶ ansible-playbook generate_inventory.yaml -i inventory/sample/hosts.yaml

PLAY [Generate inventory] ***********************************************************************************************************************************************************************************

TASK [Set some facts] ***************************************************************************************************************************************************************************************
ok: [setup]

TASK [Create group_vars/all directory] **********************************************************************************************************************************************************************
ok: [setup]

TASK [Read infra default vars] ******************************************************************************************************************************************************************************
ok: [setup]

TASK [Get inventory_vars content] ***************************************************************************************************************************************************************************
fatal: [setup]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'None' has no attribute 'download_inventory_vars'\n\nThe error appears to be in '/home/coprs/Documents/2-OPS/rs-config/infrastructure/generate_inventory.yaml': line 18, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n    - name: Get inventory_vars content\n      ^ here\n"}

PLAY RECAP **************************************************************************************************************************************************************************************************
setup                      : ok=3    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0 

Expected Behavior: The file hosts.yamlshall be generated or updated.

Steps To Reproduce: Follow the deployment quickstart procedure. The error comes at step 5: Generate or download the inventory variables

Test execution artefacts (i.e. logs, screenshots…) Format of inventory/sample/host_vars/setup/main.yaml file used for deployment:

# Platform domain name
platform_domain_name: DOMAIN_NAME

# Stash license to get before the stash operator deployment
stash_license: |
  -----BEGIN CERTIFICATE-----
  Get a license here: https://license-issuer.appscode.com/?p=stash-community
  -----END CERTIFICATE-----

vault:
  download_inventory_vars: true
  path: NC
  token: NC
  upload_backup: false
  upload_existing: false
  url: https://NC

# S3 credential to be reused accross apps
s3:
  endpoint: NC
  region: NC
  secret_key: NC
  access_key: NC

# /!\ Define the issuer related to your certificate provider.
# The issuer represents your certificate authority.
# Refer to cert-manager documentation to create your issuer
# https://cert-manager.io/docs/configuration/
# For more explanations about cert-manager issuers, reach to the following link.
# https://cert-manager.io/docs/concepts/issuer/
ingress_tls_certificate_issuer_spec:
~                                      

Check of prerequisites:

cat /etc/os-release NAME="Ubuntu" VERSION="20.04.5 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.5 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal

./safescale -version

= v22.06.0

openstacksdk==0.103.0

= v0.12.0

qemu-system-x86_64 -version QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.24) Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

= v4.2.1

packer --version 1.8.6

= v1.7.8

python3 --version Python 3.8.10 Installed

python3 -m pip --version pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8) Installed

git version git version 2.25.1 Installed

jq --version jq-1.6 Installed

sudo apt list cloud-image-utils Listing... cloud-image-utils/focal,focal,now 0.31-7-gd99b2d76-0ubuntu1 all [installed] Installed

ansible==3.4.0 ansible-base==2.10.11 ==3.4.0

collections/kubespray/requirements.txt ansible-base==2.10.11 cryptography==2.8 Jinja2==2.11.3 netaddr==0.7.19 pbr==5.4.4 jmespath==0.9.5 ruamel.yaml==0.16.10 ruamel.yaml.clib==0.2.4 MarkupSafe==1.1.1

ansible-galaxy collection install kubernetes.core 1.2.1
kubernetes.core 2.3.2
openstack.cloud 1.4.0
openstack.cloud 1.10.0

Bug Generic Definition of Ready (DoR)

• [X] The affect version in which the bug has been found is mentioned
• [X] The context and environment of the bug is detailed
• [X] The description of the bug is clear and unambiguous
• [X] The procedure (steps) to reproduce the bug is clearly detailed
• [ ] The tested User Story / features is linked to the bug if available
• [X] Logs are attached if available
• [ ] A data set attached if available

Bug Generic Definition of Done (DoD)

• [ ] the modification implemented (the solution to fix the bug) is described in the bug.
• [ ] Unit tests & Continuous integration performed - Test results available - Structural Test coverage reported by SONAR
• [ ] Code committed in GIT with right tag or Analysis/Trade Off documentation up-to-date in reference-system-documentation repository
• [ ] Code is compliant with coding rules (SONAR Report as evidence)
• [ ] Acceptance criteria of the related User story are checked and Passed

[LAQU156]

IVV_CCB_2023_w08 : Moved into "Accepted CS" for further analysis, Priority major

[Woljtek]

Here the script to check the requirements: https://app.zenhub.com/files/398313496/52a8c1de-726d-435a-aaba-bddd4a0a2893/download

[LAQU156]

CS_CCB_2023_w08 : Moved into "Product Backlog", high priority

[eroan-marie]

I couldn't reproduce the issue with the version you used. But the error message seems to says there is a problem with the word "vault" inside your configuration maybe this part has a typo like an invisible character, you can check it using the following command: cat -A inventory/sample/host_vars/setup/main.yaml You could also launch the playbook with a high verbosity : ansible-playbook generate_inventory.yaml -i inventory/mycluster/hosts.yaml -vvv

[Woljtek]

The cat -A command shows unexpected characters that yamllint accepts.

I dont have any idea to prevent End User to this kind of error.

[Woljtek]

The first error hidden this one:

TASK [Get inventory_vars content] *********************************************************************************************************************************************************************************
task path: /home/coprs/Documents/2-OPS/rs-config/infrastructure/generate_inventory.yaml:18
Loading collection community.hashi_vault from /home/coprs/.local/lib/python3.8/site-packages/ansible_collections/community/hashi_vault
fatal: [setup]: FAILED! => {
    "msg": "An unhandled exception occurred while running the lookup plugin 'community.hashi_vault.hashi_vault'. Error was a <class 'requests.exceptions.InvalidHeader'>, original message: Header part ('XXX') from {'X-Vault-Token': 'XXXX'} must be of type str or bytes, not <class 'ansible.parsing.yaml.objects.AnsibleUnicode'>"
}

PLAY RECAP ********************************************************************************************************************************************************************************************************
setup                      : ok=3    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0  

Do have any idea of the root cause?

[eroan-marie]

I can't seem to find an explanation for this issue. Did you try to execute the playbook with -vvv ? Maybe there will be more information

[eroan-marie]

You could also try to add the following block before the task "Get inventory_vars content" in generated_inventory.yaml and check if the token is written as expected

   - name: debug
      ansible.builtin.debug:
        msg: Token = {{ vault.token }}

[LAQU156]

CS_CCB_2023_w10 : Action @Woljtek : test what is suggested by @eroan-marie

[Woljtek]

I made the test. It gives a new error:

TASK [Get inventory_vars content] *********************************************************************************************************************************************************************************
fatal: [setup]: FAILED! => {"msg": "An unhandled exception occurred while running the lookup plugin 'community.hashi_vault.hashi_vault'. Error was a <class 'requests.exceptions.InvalidHeader'>, original message: Header part ('myToken') from {'X-Vault-Token': 'myToken'} must be of type str or bytes, not <class 'ansible.parsing.yaml.objects.AnsibleUnicode'>"}

[nleconte-csgroup]

@Woljtek is it working now or are you still getting errors ?

[pcuq-ads]

@Woljtek , please propose a conclusion for that ticket ?

[Woljtek]

I miss the question of @nleconte-csgroup .

No; it doesn't, the issue is still here.

I found an acceptable WA.

• Download the inventory_vars in JSON format from the VAULT keystore
• Convert the JSON to YAML: <python3 -c 'import sys, yaml, json; print(yaml.dump(json.loads(sys.stdin.read())))' inventory-vars.yaml>
• Replace the old inventory vars by the new one
• Replace app and safescale path

I propose to decrease the priority to minor and handle this point after the V2 delivery. @pcuq-ads Do you agree ?

[pcuq-ads]

RSRRv2_SystemCCB : need some "how to" documentation to explain how to avoid this issue during deployment.

[nleconte-csgroup]

RSRRv2_CS-FR_CCB : Looks like the issue is caused by the user's mishandling here. However, it will be documented in infrastructure version 1.6.0

[nleconte-csgroup]

Delivered in infrastructure 1.6.0-rc1 (https://github.com/COPRS/infrastructure/releases)

[pcuq-ads]

SYS_CCB_w29 : The documentation has been updated on release 1.6.0-rc1.