Cluster installation failed in disconnected 4.10.24

Bug description

Hi Team,

We are trying to install cluster by using current crucible playbooks but we hit an issue with AI v2.1 for pull-secret.txt.

our deploy_prerequisites.yml run's fine and every service are running good on bastion nodes, But while ruining for cluster deployment it is failing in create_cluster role and giving below error.

Requesting you to please have a look and if possible let me know the WA for now.

OpenShift version

other (provide in the description)

Assisted Installer version

v2.1.1

Relevant log output

TASK [create_cluster : debug] ***************************************************************************************************************************************************************************************************************
task path: /root/crucible/roles/create_cluster/tasks/main.yml:40
Thursday 28 July 2022  22:01:28 +0900 (0:00:00.028)       0:00:12.641 *********
ok: [bastion] =>
  msg: |-
    {
        "additional_ntp_source": "172.90.12.240",
        "api_vip": "172.90.12.241",
        "base_dns_domain": "nfvi.localdomain",
        "cluster_network_cidr": "10.128.0.0/14",
        "cluster_network_host_prefix": 23,
        "high_availability_mode": "Full",
        "http_proxy": "",
        "https_proxy": "",
        "name": "nec",
        "network_type": "OVNKubernetes",
        "no_proxy": "",
        "openshift_version": "4.10",
        "pull_secret": <redacted>,
        "schedulable_masters": true,
        "service_network_cidr": "172.30.0.0/16",
        "ssh_public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC2vk8cbOF3oDs+D6mio5gHNf9BEbbi1m08ZKzR7VMlz/VCzDNTBC8CYVzWCvDxfqpRa1I2mDP5DuYJojcrGqNr/8fY1c82IATMOh1hUrrdcbVX0PAfA9BUYVnpge1TF0CV7YE3mLPhZj6UdwV7RrQeARIFtl22A1uVr0qRRquK2qZm792/BdbH7W7WLfaXMiE6nIr8H2Rvv2HY5yvLNW7SuM0cx3w1lQlyhPVcgFfBoYRHyo7wGPWQp7gH3G4lg2K1+sqRT7qW2CDc4JOW3/F76ZJP47zoVwcHQtUuUYRaGndEPbRCFCM9/AJvGIx82mBrA1qk9P/6Pdj68vIPyd6Ufnd5LFYL4vCunexdYh/JJOhh94AYQKWYy4eYk62AuwMcPFjaJyO84DJhQ8C+9hvnXC7iPRw464TXYjn6Xi596nO4FCFwkKXzwxf3ut/c6mbP/AANVxE7oLbSHPxZdNll0+KilY+xCizkQK5K26PEAonTrE67OynYOjKA1lV/XlrJflXR9AnCq2v/4/DCfnnhMtYrZsEVuB2wYvoMMZPN6Mki/c/TsQGFN0YBu4AeaDPnhSMqIRthxYRgowkzGJ9qnX9MpWswnK55Kbz/7wXMej77M0UXbAoMY/TDMUUQg9LpEI1TsJwbMYuCLiepwWZVIrHfAaeFyoINCgj/QrXBjw==",
        "vip_dhcp_allocation": false
    }

TASK [create_cluster : Create cluster] ******************************************************************************************************************************************************************************************************
task path: /root/crucible/roles/create_cluster/tasks/main.yml:44
Thursday 28 July 2022  22:01:28 +0900 (0:00:00.030)       0:00:12.671 *********
fatal: [bastion]: FAILED! => changed=false
  connection: close
  content: |-
    {"code":"400","href":"","id":400,"kind":"Error","reason":"pull secret for new cluster is invalid: pull secret must contain auth for \"docker.io\""}
  content_length: '148'
  content_type: application/json
  date: Thu, 28 Jul 2022 13:01:28 GMT
  elapsed: 0
  json:
    code: '400'
    href: ''
    id: 400
    kind: Error
    reason: 'pull secret for new cluster is invalid: pull secret must contain auth for "docker.io"'
  msg: 'Status code was 400 and not [201]: HTTP Error 400: Bad Request'
  redirected: false
  status: 400
  url: http://5gc-bstn01.nec.nfvi.localdomain:8090/api/assisted-install/v2/clusters

PLAY RECAP **********************************************************************************************************************************************************************************************************************************
bastion                    : ok=52   changed=5    unreachable=0    failed=1    skipped=23   rescued=0    ignored=0

Thursday 28 July 2022  22:01:28 +0900 (0:00:00.566)       0:00:13.238 *********
===============================================================================
validate_inventory : Ensure service hosts are reachable ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2.08s
/root/crucible/roles/validate_inventory/tasks/network.yml:11 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
generate_ssh_key_pair : Copy SSH Key files to bastion -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.46s
/root/crucible/roles/generate_ssh_key_pair/tasks/main.yml:43 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.41s
/root/crucible/playbooks/generate_ssh_key_pair.yml:2 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
generate_ssh_key_pair : Generate an OpenSSH rsa keypair ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 1.05s
/root/crucible/roles/generate_ssh_key_pair/tasks/main.yml:21 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
generate_ssh_key_pair : Fetch SSH Key ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 0.88s
/root/crucible/roles/generate_ssh_key_pair/tasks/main.yml:26 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
create_cluster : Create cluster ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 0.57s
/root/crucible/roles/create_cluster/tasks/main.yml:44 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
generate_ssh_key_pair : Remove temporary ssh_keys folder /tmp/ssh_key_pair ----------------------------------------------------------------------------------------------------------------------------------------------------------- 0.42s
/root/crucible/roles/generate_ssh_key_pair/tasks/main.yml:2 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
generate_ssh_key_pair : Make SSH Key folder ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 0.37s
/root/crucible/roles/generate_ssh_key_pair/tasks/main.yml:37 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
generate_ssh_key_pair : Make key dir ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.36s
/root/crucible/roles/generate_ssh_key_pair/tasks/main.yml:15 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
validate_inventory : Assert all nodes have all required vars ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.31s
/root/crucible/roles/validate_inventory/tasks/cluster.yml:32 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
generate_ssh_key_pair : Make sure /root/crucible//fetched exists --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.21s
/root/crucible/roles/generate_ssh_key_pair/tasks/main.yml:7 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
validate_inventory : Assert that all nodes have BMC credentials set ------------------------------------------------------------------------------------------------------------------------------------------------------------------ 0.18s
/root/crucible/roles/validate_inventory/tasks/secrets.yml:22 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
generate_ssh_key_pair : Distribute public key to all hosts --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.13s
/root/crucible/roles/generate_ssh_key_pair/tasks/main.yml:54 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
validate_inventory : Get KVM BMC username password combinations ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.12s
/root/crucible/roles/validate_inventory/tasks/vms.yml:38 -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
validate_inventory : Assert that all credentials for the disconnected registry are set ----------------------------------------------------------------------------------------------------------------------------------------------- 0.12s
/root/crucible/roles/validate_inventory/tasks/secrets.yml:2 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
validate_inventory : include_tasks --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.11s
/root/crucible/roles/validate_inventory/tasks/main.yml:58 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
validate_inventory : Assert required vars are correctly typed ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 0.11s
/root/crucible/roles/validate_inventory/tasks/cluster.yml:67 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Check vendors role exists ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 0.09s
/root/crucible/roles/validate_inventory/tasks/vendor.yml:11 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
validate_inventory : Record successful validation on all hosts ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.09s
/root/crucible/roles/validate_inventory/tasks/main.yml:74 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
validate_inventory : include_tasks --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.09s
/root/crucible/roles/validate_inventory/tasks/main.yml:16 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[root@5gc-bstn01 crucible]#

[root@5gc-bstn01 crucible]# cat /tmp/pull-secret.json
{
  "auths": {
    "cloud.openshift.com": {
      "auth": <redacted>,
      "email": "shashank@india.nec.com"
    },
    "quay.io": {
      "auth":  <redacted>,
      "email": "shashank@india.nec.com"
    },
    "registry.connect.redhat.com": {
      "auth": <redacted>,
      "email": "shashank@india.nec.com"
    },
    "registry.redhat.io": {
      "auth": <redacted>,
      "email": "shashank@india.nec.com"
    },
    **"registry.nec.nfvi.localdomain:5000": {
      "auth": <redacted>,
      "email": "root@registry.nec.nfvi.localdomain"**
    }
  }
}
[root@5gc-bstn01 crucible]#

Inventory file

all:
  vars:
    ##################################
    # Assisted Install Configuration #
    ##################################
    # These options configure Assisted Installer and the resulting cluster
    # https://generator.swagger.io/?url=https://raw.githubusercontent.com/openshift/assisted-service/58a6abd5c99d4e41d939be89cd0962433849a861/swagger.yaml
    # See section: cluster-create-params

    # Cluster name and dns domain combine to give the cluster namespace that will contain OpenShift endpoints
    # e.g. api.clustername.example.lab, worker1.clustername.example.lab
    cluster_name: nec
    base_dns_domain: nfvi.localdomain

    # OpenShift version (4.6.16, 4.7.33, 4.8.14 or 4.9.11)
    openshift_full_version: 4.10.24
    ai_version: v2.1.1
    supported_ocp_versions:
      - 4.6.16
      - 4.7.33
      - 4.8.12
      - 4.9.29
      - 4.10.24
    os_images:
        - openshift_version: '4.10'
          cpu_architecture: x86_64
          url: "http://{{ hostvars['http_store']['ansible_host'] }}/discovery/rhcos-4.10.16-x86_64-live.x86_64.iso"
          rootfs_url: "http://{{ hostvars['http_store']['ansible_host'] }}/discovery/rhcos-4.10.16-x86_64-live-rootfs.x86_64.img"
          version: 410.84.202201251210-0
    release_images:
        - openshift_version: '4.10'
          cpu_architecture: x86_64
          url: registry.nec.nfvi.localdomain:5000/ocp4/openshift4:4.10.24-x86_64
          version: 4.10.24
          default: true
    assisted_service_image_repo_url: registry.nec.nfvi.localdomain:5000/ocpmetal

    # Virtual IP addresses used to access the resulting OpenShift cluster
    api_vip: IP # the IP address to be used for api.clustername.example.lab and api-int.clustername.example.lab
    ingress_vip: IP # the IP address to be used for *.apps.clustername.example.lab

    ## Allocate virtual IPs via DHCP server. Equivalent to the vip_dhcp_allocation configuration option of Assisted Installer
    vip_dhcp_allocation: false

    # The subnet on which all nodes are (or will be) accessible.
    machine_network_cidr: 0.0.0.0/24

    # The IP address pool to use for service IP addresses
    service_network_cidr: 172.30.0.0/16

    # Cluster network settings. You are unlikely to need to change these
    cluster_network_cidr: 10.128.0.0/14 # The subnet, internal to the cluster, on which pods will be assigned IPs
    cluster_network_host_prefix: 23 # The subnet prefix length to assign to each individual node.

    # # Cluster network provider. Cannot be changed after cluster is created.
    # # The default is OpenShift SDN unless otherwise specified.
    network_type: OVNKubernetes
    # network_type: OpenShiftSDN

    ######################################
    # Prerequisite Service Configuration #
    ######################################

    # Proxy settings. These settings apply to: Assisted Installer, Day1 clusters and Day2 clusters.
    # This assumes the host where the AI runs and the OpenShift cluster share the same proxy settings.
    # http_proxy: ""
    # https_proxy: ""
    # no_proxy: ""

    # Flags to enable/disable prerequisite service setup
    # You will need to ensure alternatives are available for anything that will not be automatically set up
    setup_ntp_service: true
    setup_dns_service: false
    setup_pxe_service: false
    setup_registry_service: false # Only required for a Restricted Network installation
    setup_http_store_service: true
    setup_assisted_installer: true # default is true you may wish to turn it off if multiple users are using the same instance.

    # NTP Service
    # ntp_server is the address at which the NTP service is (or will be) available
    ntp_server: IP
    # ntp_server_allow is the range of IPs the NTP service will respond to
    ntp_server_allow: 0.0.0.0/24 # not required if setup_ntp_service is false

    # Mirror Registry Service parameters for a Restricted Network installation

    # use_local_mirror_registry controls if the install process uses a local container registry (mirror_registry) or not.
    # Set this to true to use the mirror registry service set up when `setup_registry_service` is true.
    use_local_mirror_registry: true

    # HTTP Store Configuration
    # ISO name must include the `discovery` directory if you have a SuperMicro machine
    discovery_iso_name: "discovery/{{ cluster_name }}/discovery-image.iso"

    # discovery_iso_server must be discoverable from all BMCs in order for them to mount the ISO hosted there.
    # It is usually necessary to specify different values for KVM nodes and/or physical BMCs if they are on different subnets.
    discovery_iso_server: "http://{{ hostvars['http_store']['ansible_host'] }}"

    ############################
    # Local File Configuration #
    ############################

    repo_root_path: /root/crucible/ # path to repository root

    # Directory in which created/updated artifacts are placed
    fetched_dest: "{{ repo_root_path }}/fetched"

    # Configure possible paths for the pull secret
    # first one found will be used
    # note: paths should be absolute
    pull_secret_lookup_paths:
      - "{{ fetched_dest }}/pull-secret.txt"
      - "{{ repo_root_path }}/pull-secret.txt"

    # Configure possible paths for the ssh public key used for debugging
    # first one found will be used
    # note: paths should be absolute
    ssh_public_key_lookup_paths:
      - "{{ fetched_dest }}/ssh_keys/{{ cluster_name }}.pub"
      - "{{ repo_root_path }}/ssh_public_key.pub"
      - ~/.ssh/id_rsa.pub

    # Set the base directory to store ssh keys
    ssh_key_dest_base_dir: /home/redhat

    # The retrieved cluster kubeconfig will be placed on the bastion host at the following location
    kubeconfig_dest_dir: /home/redhat/
    kubeconfig_dest_filename: "{{ cluster_name }}-kubeconfig"
    kubeadmin_dest_filename: "{{ cluster_name }}-kubeadmin.vault.yml"
    # You can comment out the line below if you want the kubeadmin credentials to be stored in plain text
    #kubeadmin_vault_password_file_path: "{{ repo_root_path }}/kubeadmin_vault_password_file"

    ############################
    #    LOGIC: DO NOT TOUCH   #
    # vvvvvvvvvvvvvvvvvvvvvvvv #
    ############################

    # pull secret logic, no need to change. Configure above
    local_pull_secret_path: "{{ lookup('first_found', pull_secret_lookup_paths) }}"
    pull_secret: "{{ lookup('file', local_pull_secret_path) }}"

    # ssh key logic, no need to change. Configure above
    local_ssh_public_key_path: "{{ lookup('first_found', ssh_public_key_lookup_paths) }}"
    ssh_public_key: "{{ lookup('file', local_ssh_public_key_path) }}"

    # provided mirror certificate logic, no need to change.
    local_mirror_certificate_path: "{{ (setup_registry_service == true) | ternary(
        fetched_dest + '/' + (hostvars['registry_host']['cert_file_prefix'] | default('registry')) + '.crt',
        repo_root_path + '/mirror_certificate.txt')
      }}"
    mirror_certificate: "{{ lookup('file', local_mirror_certificate_path) }}"

    openshift_version: "{{ openshift_full_version.split('.')[:2] | join('.') }}"

    is_valid_single_node_openshift_config: "{{ (groups['nodes'] | length == 1) and (groups['masters'] | length == 1) }}"

    ############################
    # ^^^^^^^^^^^^^^^^^^^^^^^^ #
    #    LOGIC: DO NOT TOUCH   #
    ############################

  children:
    bastions: # n.b. Currently only a single bastion is supported
      hosts:
        bastion:
          ansible_host:  IP # Must be reachable from the Ansible control node

    # Configuration and access information for the pre-requisite services
    # TODO: document differences needed for already-deployed and auto-deployed
    services:
      hosts:
        assisted_installer:
          ansible_host: IP
          host: 5gc-bstn01.nec.nfvi.localdomain
          port: 8090 # Do not change
          dns_servers:
            - IP

        registry_host:
          ansible_host: IP
          registry_port: 5000
          registry_fqdn: registry.nec.nfvi.localdomain # use in case of different FQDN for the cert
          cert_common_name: "{{ registry_fqdn }}"
          cert_country: US
          cert_locality: Raleigh
          cert_organization: Red Hat, Inc.
          cert_organizational_unit: Lab
          cert_state: NC

          # Configure the following secret values in the inventory.vault.yml file
          REGISTRY_HTTP_SECRET: "{{ VAULT_REGISTRY_HOST_REGISTRY_HTTP_SECRET | mandatory }}"
          disconnected_registry_user: "{{ VAULT_REGISTRY_HOST_DISCONNECTED_REGISTRY_USER | mandatory }}"
          disconnected_registry_password: "{{ VAULT_REGISTRY_HOST_DISCONNECTED_REGISTRY_PASSWORD | mandatory }}"

        dns_host:
          ansible_host: IP
          # upstream_dns: 8.8.8.8 # an optional upstream dns server
          # The following are required for DHCP setup
          # use_dhcp: true
          # use_pxe: false
          # dhcp_range_first: 10.60.0.101
          # dhcp_range_last:  10.60.0.105
          # prefix: 24
          # gateway: 10.60.0.1

        http_store:
          ansible_host: IP

        tftp_host:
          ansible_host: IP
          tftp_directory: /var/lib/tftpboot/

        ntp_host:
          ansible_host: IP

    vm_hosts:
      hosts:
        vm_host1: # Required for using "KVM" nodes, ignored if not.
          ansible_user: root
          ansible_host: IP
          host_ip_keyword: ansible_host # the varname in the KVM node hostvars which contains the *IP* of the VM
          images_dir: /extra # directory where qcow images will be placed.
          vm_bridge_ip: IP # IP for the bridge between VMs and machine network
          vm_bridge_interface:  # Interface to be connected to the bridge. DO NOT use your primary interface.
          #dns: 10.40.0.100 # DNS used by the bridge
          vm_bridge_name: caas-br
          SETUP_VM_BRIDGE: false
          # ssl cert configuration
          # sushy_fqdn: ... # use in case of different FQDN for the cert
          cert_vars_host_var_key: registry_host # Look up cert values from another host by name (excluding cert_common_name)
          # or
          # cert_country: US
          # cert_locality: Raleigh
          # cert_organization: Red Hat, Inc.
          # cert_organizational_unit: Lab
          # cert_state: NC

    # Describe the desired cluster members
    nodes:
      # A minimum of three master nodes are required. More are supported.
      # Worker nodes are not required, but if present there must be two or more.
      #
      # Node Required Vars:
      # - role
      #     - Must be either "master" or "worker", and must match the group
      #
      # - mac
      #     - The MAC address of the node, used as a hardware identifier by Assisted Installer.
      #     - The value set here will be used when creating VMs and must be unique within the network.
      #
      # - vendor
      #     - One of "Dell", "HPE", "Lenovo", "SuperMicro", "KVM", "PXE" as the supported BMC APIs.
      #     - "KVM" identifies a node as a VM to be created. If a "KVM" node is present,
      #       then a "vm_host" must be defined in the node and a host with that name must exist
      #       inside the "vm_hosts" group.
      #     - "PXE" identifies a node as a baremetal that needs to boot from PXE.
      #
      # - bmc_address
      # - bmc_user
      # - bmc_password
      #     - details for the BMC that controls the node.
      #     - Must be set to the vm_host for "KVM" nodes.
      #
      # Static IP Vars:
      #   See docs/inventory.md: Network configuration section
      #
      # Optional Vars:
      # - vm_spec
      #     - Specifications for the node:
      #          - cpu_cores
      #          - ram_mib
      #          - disk_size_gb
      #
      # - installation_disk_path
      #     - The value set here will be used by Assisted Installer as the installation disk device
      #       for a given host.
      #     - The value must be a path to the disk device, e.g. /dev/sda
      #     - If not specified, Assisted Installer will pick the first enumerated disk device for a
      #       given host.
      vars:
        # Set the login information for any BMCs. Note that these will be SET on the vm_host virtual BMC.
        bmc_user: "{{ VAULT_NODES_BMC_USER | mandatory }}"
        bmc_password: "{{ VAULT_NODES_BMC_PASSWORD | mandatory }}"
        dns1: "IP"
        gateway: "IP"
        mask: 24
        network_config:
          dns_server_ips:
            - "{{ dns1 }}"
          interfaces:
            -
              addresses:
                ipv4:
                  -
                    ip: "{{ ansible_host }}"
                    prefix: "{{ mask }}"
              dhcp: false
              mac: "{{ mac }}"
              name: enp1s0
              state: up
              type: ethernet
          routes:
            -
              address: "{{ gateway }}"
              destination: 0.0.0.0/0
              interface: enp1s0
      children:
        masters:
          vars:
            role: master
            vendor: KVM # this example is a virtual control plane
            bmc_address: IP:8082 # port can be changed using sushy_tools_port on the vm_host
            vm_host: vm_host1
            vm_spec:
              cpu_cores: 16
              ram_mib: 40384
              disk_size_gb: 100
              extra_disks:
                my_extra_disk: 250
          hosts:
            super1:
              ansible_host: IP
              mac: "DE:AD:BE:EF:C0:2C"

              # # Uncomment to set custom BMC credentials for the node
              # # These variables must be set in the inventory.vault.yml file
              # bmc_user: "{{ VAULT_NODES_SUPER1_BMC_USER | mandatory }}"
              # bmc_password: "{{ VAULT_NODES_SUPER1_BMC_PASSWORD | mandatory }}"

            super2:
              ansible_host: IP
              mac: "DE:AD:BE:EF:C0:2D"

            super3:
              ansible_host: IP
              mac: "DE:AD:BE:EF:C0:2E"

#        workers:
#          vars:
#            role: worker
#            vendor: Dell # This example uses baremetal worker nodes
#          hosts:
#            worker1:
#              ansible_host: 10.60.0.104
#              bmc_address: 172.28.11.25
#              mac: 3C:FD:FE:78:AB:03

              # # Uncomment to set custom BMC credentials for the node
              # # These variables must be set in the inventory.vault.yml file
              # bmc_user: "{{ VAULT_NODES_WORKER1_BMC_USER | mandatory }}"
              # bmc_password: "{{ VAULT_NODES_WORKER1_BMC_PASSWORD | mandatory }}"

              # # Uncomment to set an alternate installation disk device for the node
              # installation_disk_path: /dev/sdb

#            worker2:
#              ansible_host: 10.60.0.105
#              bmc_address: 172.28.11.26
#              mac: 3C:FD:FE:78:AB:04
#
              # # Uncomment to set an alternate installation disk device for the node
              # installation_disk_path: /dev/sdb

Required statements

[X] I have removed all sensitive details from the attached logs and inventory files.
[X] I acknowledge that Red Hat does not provide commercial support for the content of this repository.
[X] I acknowledge that any assistance is offered purely on a best-effort basis, as resource permits.

redhat-partner-solutions / crucible

Cluster installation failed in disconnected 4.10.24 #166