search

okd-project / okd

The self-managing, auto-upgrading, Kubernetes distribution for everyone
https://okd.io
Apache License 2.0
1.76k stars 297 forks source link

Installation of OKD 4.5 fails on vSphere with creation of vsphere_tag_category #279

Closed aholen closed 3 years ago

aholen commented 4 years ago

Describe the bug Trying to install okd 4.5 as described https://docs.okd.io/latest/installing/installing_vsphere/installing-vsphere-installer-provisioned.html - but the provisioning fails when trying to create categories in vsphere.

The user myusername@yyy.com has all the tags/categories-permissions in vsphere.

Version

./openshift-install version
./openshift-install 4.5.0-0.okd-2020-07-14-153706-ga
built from commit 290e3b1de6096ecef2133fb071ff3a71c9c78594
release image quay.io/openshift/okd@sha256:a2d5ef5e3884b125761a5fae1f7e89748eb6880d8ef5c7da4d11a1f661251532

vSphere Client version 6.7.0.44000

How reproducible Reproducable 100% in my environment.

Log bundle ./openshift-install create cluster ? SSH Public Key /Users/REDACTED/.ssh/id_rsa.pub ? Platform vsphere ? vCenter xxx.yyy.com ? Username myusername@yyy.com ? Password [? for help] *** INFO Connecting to vCenter xxx.yyy.com INFO Defaulting to only available datacenter: H ? Cluster Cluster-H ? Default Datastore REDACTED ? Network REDACTED ? Virtual IP Address for API 10.xx.yy.27 ? Virtual IP Address for Ingress 10.xx.zz.84 ? Base Domain os.nnn.com ? Cluster Name dev ? Pull Secret [? for help] ** INFO Obtaining RHCOS image file from 'https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/32.20200629.3.0/x86_64/fedora-coreos-32.20200629.3.0-vmware.x86_64.ova?sha256=172f299a3e28be360740ff437a5ea9bfc246f52ea8f313d4138c5d16fd4b11e1' INFO The file was found in cache: /Users/myusername/Library/Caches/openshift-installer/image_cache/062bfe3785d26fa220e2e6e72d1b3562. Reusing... INFO Creating infrastructure resources... ERROR ERROR Error: could not create category: POST https://xxx.yyy.com/rest/com/vmware/cis/tagging/category: 403 Forbidden ERROR ERROR on ../../../../var/folders/gc/_10kfk2x0471h9pv4jmmyd340000gn/T/openshift-install-828305376/main.tf line 54, in resource "vsphere_tag_category" "category": ERROR 54: resource "vsphere_tag_category" "category" { ERROR ERROR FATAL failed to fetch Cluster: failed to generate asset "Cluster": failed to create cluster: failed to apply Terraform: failed to complete the change

abaxo commented 4 years ago

I'm able to run that here using effectively a full local admin account from a number of 4.5 releases. It looks like (because of the 403 Forbidden on the POST request to create the tagging/category resource) that the account you are using to auth with doesn't have the correct permissions for create categories. The permissions needed are documented here: https://docs.okd.io/latest/installing/installing_vsphere/installing-vsphere-installer-provisioned-customizations.html#installation-vsphere-installer-infra-requirements_installing-vsphere-installer-provisioned-customizations

Worth double checking that your user holds the following permissions:

vSphere Tagging
Assign or Unassign vSphere Tag
Assign or Unassign vSphere Tag on Object
Create vSphere Tag
Create vSphere Tag Category
Delete vSphere Tag
Delete vSphere Tag Category
Edit vSphere Tag
Edit vSphere Tag Category
Modify UsedBy Field For Category
Modify UsedBy Field For Tag

Hope this helps, Craig

aholen commented 4 years ago

@abaxo Hi, thanks - I've checked those, and the only one I'm missing is "Assign or Unassign vSphere Tag on Object" - perhaps that's a vSphere 7.x-permission?

abaxo commented 4 years ago

@abaxo Hi, thanks - I've checked those, and the only one I'm missing is "Assign or Unassign vSphere Tag on Object" - perhaps that's a vSphere 7.x-permission?

Ah yes, they could be. I think if you have the 'Create vSphere Tag Category' permission (for your specific issue) then that should sort it out. Something that I have seen with vSphere before, though not with Openshift, is that you can have permissions that conflict with each other if your user is a member of a group, it is possible those group permissions conflict. When I ran into this one group supplied the correct permissions (funnily enough around tagging, too) but the other group that the user was part of conflicted, and denied that permission. It might be worth trying having the user outside of any groups that you may have to prove the permissions then work backwards from there.

vrutkovs commented 4 years ago

Has this been resolved? Do we need to update the docs to mention new permissions or it has a user misconfiguration?

josedavireges commented 4 years ago

I have a same issue! DEBUG If you ever set or change modules or backend configuration for Terraform, DEBUG rerun this command to reinitialize your working directory. If you forget, other DEBUG commands will detect it and remind you to do so if necessary. DEBUG data.vsphere_datacenter.datacenter: Refreshing state... DEBUG data.vsphere_network.network: Refreshing state... DEBUG data.vsphere_datastore.datastore: Refreshing state... DEBUG data.vsphere_compute_cluster.cluster: Refreshing state... DEBUG vsphere_tag_category.category: Creating... ERROR ERROR Error: could not create category: POST https://**********/rest/com/vmware/cis/tagging/category: 403 Forbidden ERROR ERROR on ../../tmp/openshift-install-536342876/main.tf line 54, in resource "vsphere_tag_category" "category": ERROR 54: resource "vsphere_tag_category" "category" { ERROR ERROR FATAL failed to fetch Cluster: failed to generate asset "Cluster": failed to create cluster: failed to apply Terraform: failed to complete the change

vrutkovs commented 4 years ago

Closing it as due to insufficient info

aholen commented 4 years ago

This has not been resolved. The docs should be updated to correct permissions, or at least map which permissions are needed on Vsphere 6.x oand Vsphere 7.x

aholen commented 4 years ago

/reopen

openshift-ci-robot commented 4 years ago

@aholen: Reopened this issue.

In response to [this](https://github.com/openshift/okd/issues/279#issuecomment-690102308): >/reopen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
vrutkovs commented 4 years ago

The docs should be updated to correct permissions

Which permissions need to be added then?

aholen commented 4 years ago

I don't know, maybe @abaxo could answer that. I have compared my users privileges with the Administrator-account on vpshere, and can confirm that I have all the permissions related to tags and category. @abaxo Did you test againt vSphere 6.x or 7.x?

magick93 commented 4 years ago

I'm not sure, but this could be the cause:

Something that I have seen with vSphere before, though not with Openshift, is that you can have permissions that conflict with each other if your user is a member of a group, it is possible those group permissions conflict. When I ran into this one group supplied the correct permissions (funnily enough around tagging, too) but the other group that the user was part of conflicted, and denied that permission. It might be worth trying having the user outside of any groups that you may have to prove the permissions then work backwards from there.

That is, check if youre in two, or more, vsphere groups/roles that may conflict.

Also, try making a new account, with god permissions, and try again - not the same account.

aholen commented 4 years ago

I have dug more into this now, and was able to create tags with the installer when my user was given global permissions, instead of only on our vsphere-cluster. The doc should be updated with this specifcations, according to https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.security.doc/GUID-2199584C-B422-4EEF-9340-5449E1FB7DAE.html

I ran into a new problem, and will create a new issue with this:

INFO Obtaining RHCOS image file from 'https://releases-art-rhcos.svc.ci.openshift.org/art/storage/releases/rhcos-4.6/46.82.202008260918-0/x86_64/rhcos-46.82.202008260918-0-vmware.x86_64.ova?sha256=f1a193fd132392d8004fcb96b5594d3373069d1d32494f8d4147f4bc1478ed9b' INFO The file was found in cache: /Users/user/Library/Caches/openshift-installer/image_cache/5eebf72ce4c47ff0816d4a452640fd0f. Reusing... INFO Creating infrastructure resources... ERROR ERROR Error: failed to find provided vSphere objects: cluster '/X15/host/YYYCluster-X15' not found ERROR ERROR on ../../../../var/folders/gc/_10kfk2x0471h9pv4jmmyd340000gn/T/openshift-install-409444778/main.tf line 43, in resource "vsphereprivate_import_ova" "import": ERROR 43: resource "vsphereprivate_import_ova" "import" { ERROR ERROR FATAL failed to fetch Cluster: failed to generate asset "Cluster": failed to create cluster: failed to apply Terraform: failed to complete the change

vrutkovs commented 4 years ago

INFO Obtaining RHCOS image file from 'releases-art-rhcos.svc.ci.openshift.org/art/storage/releases/rhcos-4.6/46.82.202008260918-0/x86_64/rhcos-46.82.202008260918-0-vmware.x86_64.ova?sha256=f1a193fd132392d8004fcb96b5594d3373069d1d32494f8d4147f4bc1478ed9b'

You're using a wrong installer, which pulls in RHCOS instead of FCOS

openshift-bot commented 3 years ago

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale