Install error in "Retrieve list of schedulable nodes matching selector" -- oc get node... --> Error: unknown flag: --field-selector

[AllezWiggo]

Description

Provide a brief description of your issue here. For example:

Error during multinode, single master install

Version

Please put the following version information in the code block indicated below.

Place the output between the code block below:

ansible 2.4.2.0
openshift-ansible-3.10.0-0.1.0-72-gdd5b4d1

Steps To Reproduce

1. ansible-playbook -i openshift-ansible/inventory/hosts.ocpdemo.local openshift-ansible/playbooks/prerequisites.yml
2. ansible-playbook -i openshift-ansible/inventory/hosts.ocpdemo.local openshift-ansible/playbooks/deploy_cluster.yml

Expected Results

Describe what you expected to happen.

OCP cluster successfully setup.

Observed Results

Error

Failure summary:

  1. Hosts:    master.ocpdemo.local
     Play:     Service Catalog
     Task:     Retrieve list of schedulable nodes matching selector
     Message:  {u'cmd': u'/bin/oc get node --selector=region=infra --field-selector=spec.unschedulable!=true -o json -n default', u'returncode': 1, u'results': [{}], u'stderr': u'Error: unknown flag: --field-selector\n\n\nUsage:\n  oc get [(-o|--output=)json|yaml|wide|custom-columns=...|custom-columns-file=...|go-template=...|go-template-file=...|jsonpath=...|jsonpath-file=...] (TYPE [NAME | -l label] | TYPE/NAME ...) [flags] [options]\n\nExamples:\n  # List all pods in ps output format.\n  oc get pods\n  \n  # List a single replication controller with specified ID in ps output format.\n  oc get rc redis\n  \n  # List all pods and show more details about them.\n  oc get -o wide pods\n  \n  # List a single pod in JSON output format.\n  oc get -o json pod redis-pod\n  \n  # Return only the status value of the specified pod.\n  oc get -o template pod redis-pod --template={{.currentState.status}}\n\nOptions:\n      --all-namespaces=false: If present, list the requested object(s) across all namespaces. Namespace in current context is ignored even if specified with --namespace.\n      --allow-missing-template-keys=true: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats.\n      --experimental-use-openapi-print-columns=false: If true, use x-kubernetes-print-column metadata (if present) from openapi schema for displaying a resource.\n      --export=false: If true, use \'export\' for the resources.  Exported resources are stripped of cluster-specific information.\n  -f, --filename=[]: Filename, directory, or URL to files identifying the resource to get from a server.\n      --ignore-not-found=false: Treat "resource not found" as a successful retrieval.\n      --include-extended-apis=true: If true, include definitions of new APIs via calls to the API server. [default true]\n  -L, --label-columns=[]: Accepts a comma separated list of labels that are going to be presented as columns. Names are case-sensitive. You can also use multiple flag options like -L label1 -L label2...\n      --no-headers=false: When using the default or custom-column output format, don\'t print headers (default print headers).\n  -o, --output=\'\': Output format. One of: json|yaml|wide|name|custom-columns=...|custom-columns-file=...|go-template=...|go-template-file=...|jsonpath=...|jsonpath-file=... See custom columns [http://kubernetes.io/docs/user-guide/kubectl-overview/#custom-columns], golang template [http://golang.org/pkg/text/template/#pkg-overview] and jsonpath template [http://kubernetes.io/docs/user-guide/jsonpath].\n      --raw=\'\': Raw URI to request from the server.  Uses the transport specified by the kubeconfig file.\n  -R, --recursive=false: Process the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory.\n      --schema-cache-dir=\'~/.kube/schema\': If non-empty, load/store cached API schemas in this directory, default is \'$HOME/.kube/schema\'\n  -l, --selector=\'\': Selector (label query) to filter on, supports \'=\', \'==\', and \'!=\'.\n  -a, --show-all=true: When printing, show all resources (false means hide terminated pods.)\n      --show-kind=false: If present, list the resource type for the requested object(s).\n      --show-labels=false: When printing, show all labels as the last column (default hide labels column)\n      --sort-by=\'\': If non-empty, sort list types using this field specification.  The field specification is expressed as a JSONPath expression (e.g. \'{.metadata.name}\'). The field in the API resource specified by this JSONPath expression must be an integer or a string.\n      --template=\'\': Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates [http://golang.org/pkg/text/template/#pkg-overview].\n  -w, --watch=false: After listing/getting the requested object, watch for changes.\n      --watch-only=false: Watch for changes to the requested object(s), without listing/getting first.\n\nUse "oc options" for a list of global command-line options (applies to all commands).\n\n', u'stdout': u''}

For long output or logs, consider using a gist

Additional Information

Provide any additional information which may help us diagnose the issue.

CentOS Linux release 7.4.1708 (Core)

[AllezWiggo]

Inventory file:

[OSEv3:children]
masters
nodes
etcd

[OSEv3:vars]
# if your target hosts are Fedora uncomment this
#ansible_python_interpreter=/usr/bin/python3
ansible_user=centos
# If ansible_user is not root, ansible_become must be set to true and the
# user must be configured for passwordless sudo
ansible_become=yes
openshift_deployment_type=origin
openshift_release=3.7
osm_cluster_network_cidr=10.128.0.0/14
openshift_portal_net=172.30.0.0/16
osm_host_subnet_length=9
# localhost likely doesn't meet the minimum requirements
openshift_disable_check=disk_availability,memory_availability

[masters]
master.ocpdemo.local

[etcd]
master.ocpdemo.local

[nodes]
master.ocpdemo.local openshift_schedulable=true openshift_node_labels="{'region': 'infra', 'zone': 'default'}"
node1.ocpdemo.local
node2.ocpdemo.local

[clcollins]

Receiving the same error with release-3.9, RHEL Atomic hosts, ansible 2.4.3.0:

TASK [openshift_master : Ensure that Template Service Broker has nodes to run on] *******************************************************************************************
fatal: [masternode-01]: FAILED! => {"msg": "Unexpected templating type error occurred on ({{  __schedulable_nodes_matching_selector['results']['results'][0]['items'] | default([]) | length != 0 }}): object of type 'builtin_function_or_method' has no len()"}

Looks like it ties back to this Ansible module: https://github.com/openshift/openshift-ansible/blob/release-3.9/roles/lib_openshift/src/ansible/oc_obj.py

Manually running the oc get nodes command with the same flags produces the same result:

oc get node --selector=region=infra --field-selector=spec.unschedulable!=true -o json -n default
Error: unknown flag: --field-selector

Usage:
  oc get [(-o|--output=)json|yaml|wide|custom-columns=...|custom-columns-file=...|go-template=...|go-template-file=...|jsonpath=...|jsonpath-file=...] (TYPE [NAME | -l label] | TYPE/NAME ...) [flags] [options]

<snip>

[abutcher]

The master branch's oc_obj module is using --field-selector which was introduced in the 3.9 client so using the master branch with other releases like 3.7 is not compatible. This isn't the only reason but it's the most recent incompatibility.

Using the release-3.7 branch with 3.7 is the recommended approach. We've also had a pretty big docs bug recommending use of the master branch which is now corrected here https://docs.openshift.org/latest/install_config/install/host_preparation.html#preparing-for-advanced-installations-origin and matches our README, "If you're running from the openshift-ansible master branch we can only guarantee compatibility with the newest origin releases in development."

[abutcher]

Same goes for release-3.9 and any release that isn't 3.9. If you're using release-3.9 with 3.9 packages and getting this error then it's a definite bug.

[clcollins]

Well, that likely explains my issue. Thanks for the heads up.

[AllezWiggo]

Ah, so openshift_release in inventory needs to be in sync with ansible repo used. Will correct this and try. Thanks.

[AllezWiggo]

I've commented "openshift_release" in inventory file and still getting the same error. If no version is specified does it use the latest? What value does openshift_release accept to install latest version? I also tried giving:

openshift_release=v3.10
openshift_pkg_version=-3.10.0

But still getting the error:

TASK [openshift_version : fail] *********************************************************************************************************************************************************************
fatal: [master.ocpdemo.local]: FAILED! => {"changed": false, "msg": "Package 'origin-3.10*' not found"}
        to retry, use: --limit @/home/centos/openshift-ansible/playbooks/deploy_cluster.retry

[vrutkovs]

If no version is specified does it use the latest?

It would use the latest found in the repos. What's the output of yum list origin?

But still getting the error:

Neither 3.9 nor 3.10 are released and not available in CentOS repos

[AllezWiggo]

Ah ok, was able to proceed with 3.8

Installed Packages
origin.x86_64            3.7.0-1.0.7ed6862               @centos-openshift-origin

[vrutkovs]

Right, in that case you should use release-3.7 branch