travisn
commented
4 years ago @saltFisher Can you provide more details on your repro? Did you modify any of the examples? What commands exactly did you run?
Closed saltFisher closed 3 years ago
travisn
commented
4 years ago @saltFisher Can you provide more details on your repro? Did you modify any of the examples? What commands exactly did you run?
saltFisher
commented
4 years ago @travisn Hi Travis, i modified cluster.yml which changed useAllDevics from true to false, changed deviceFilter to sdc. Except this part, i didn't change anything.
operation sequence: kubectl create -f common.yaml kubectl create -f operator.yaml kubectl create -f cluster.yaml
leseb
commented
4 years ago Are you using the latest common.yaml from the repo 1.3.3? It must match your operator image.
saltFisher
commented
4 years ago @leseb Hi Sebastian，yes, i used the latest common.yaml. i also found a new phenomenon, this issue can not be recurrent 100%. i met this issue yesterday, it is ready today, i just sleep one night, not change anything. when i met this issue, i used "kubectl get crd" to check whether created this kind, it was created. Do you have any propose for me to check something to find root cause ?
$ kubectl get crd
NAME CREATED AT
cephblockpools.ceph.rook.io 2020-05-19T18:05:06Z
cephclients.ceph.rook.io 2020-05-19T18:05:06Z
cephclusters.ceph.rook.io 2020-05-19T18:05:06Z
cephfilesystems.ceph.rook.io 2020-05-19T18:05:06Z
cephnfses.ceph.rook.io 2020-05-19T18:05:06Z
cephobjectstores.ceph.rook.io 2020-05-19T18:05:06Z
cephobjectstoreusers.ceph.rook.io 2020-05-19T18:05:06Z
objectbucketclaims.objectbucket.io 2020-05-19T18:05:06Z
objectbuckets.objectbucket.io 2020-05-19T18:05:06ZDoes this show an apiVersion of ceph.rook.io/v1? Something just isn't matching up in the version. Please check your source for the manifests.
kubectl get crd cephclusters.ceph.rook.io -o yaml@travisn Sebastian, yes, i can get apiVersion of ceph.rook.io/v1, it is v1.
detail output
[root@CephOnK8S ~]# kubectl get crd cephclusters.ceph.rook.io -o yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
creationTimestamp: "2020-05-19T18:17:27Z"
generation: 1
name: cephclusters.ceph.rook.io
resourceVersion: "19973532"
selfLink: /apis/apiextensions.k8s.io/v1/customresourcedefinitions/cephclusters.ceph.rook.io
uid: 3f8b0387-1559-4f03-9fc9-a86b14cad749
spec:
conversion:
strategy: None
group: ceph.rook.io
names:
kind: CephCluster
listKind: CephClusterList
plural: cephclusters
singular: cephcluster
preserveUnknownFields: true
scope: Cluster
versions:
- additionalPrinterColumns:
- description: Directory used on the K8s nodes
jsonPath: .spec.dataDirHostPath
name: DataDirHostPath
type: string
- description: Number of MONs
jsonPath: .spec.mon.count
name: MonCount
type: string
- jsonPath: .metadata.creationTimestamp
name: Age
type: date
- description: Phase
jsonPath: .status.phase
name: Phase
type: string
- description: Message
jsonPath: .status.message
name: Message
type: string
- description: Ceph Health
jsonPath: .status.ceph.health
name: Health
type: string
name: v1
schema:
openAPIV3Schema:
properties:
spec:
properties:
annotations: {}
cephVersion:
properties:
allowUnsupported:
type: boolean
image:
type: string
cleanupPolicy:
properties:
deleteDataDirOnHosts:
pattern: ^$|^yes-really-destroy-data$
type: string
continueUpgradeAfterChecksEvenIfNotHealthy:
type: boolean
dashboard:
properties:
enabled:
type: boolean
port:
maximum: 65535
minimum: 0
type: integer
ssl:
type: boolean
urlPrefix:
type: string
dataDirHostPath:
pattern: ^/(\S+)
type: string
disruptionManagement:
properties:
machineDisruptionBudgetNamespace:
type: string
manageMachineDisruptionBudgets:
type: boolean
managePodBudgets:
type: boolean
osdMaintenanceTimeout:
type: integer
external:
properties:
enable:
type: boolean
mgr:
properties:
modules:
items:
properties:
enabled:
type: boolean
name:
type: string
mon:
properties:
allowMultiplePerNode:
type: boolean
count:
maximum: 9
minimum: 0
type: integer
volumeClaimTemplate: {}
monitoring:
properties:
enabled:
type: boolean
rulesNamespace:
type: string
network:
properties:
hostNetwork:
type: boolean
provider:
type: string
selectors: {}
placement: {}
rbdMirroring:
properties:
workers:
type: integer
removeOSDsIfOutAndSafeToRemove:
type: boolean
resources: {}
skipUpgradeChecks:
type: boolean
storage:
properties:
config: {}
deviceFilter:
type: string
devicePathFilter:
type: string
disruptionManagement:
properties:
machineDisruptionBudgetNamespace:
type: string
manageMachineDisruptionBudgets:
type: boolean
managePodBudgets:
type: boolean
osdMaintenanceTimeout:
type: integer
nodes:
items:
properties:
config:
properties:
databaseSizeMB:
type: string
encryptedDevice:
pattern: ^(true|false)$
type: string
journalSizeMB:
type: string
metadataDevice:
type: string
osdsPerDevice:
type: string
storeType:
pattern: ^(bluestore)$
type: string
walSizeMB:
type: string
deviceFilter:
type: string
devicePathFilter:
type: string
devices:
items:
properties:
config: {}
name:
type: string
type: array
name:
type: string
resources: {}
useAllDevices:
type: boolean
type: array
storageClassDeviceSets: {}
useAllDevices:
type: boolean
useAllNodes:
type: boolean
served: true
storage: true
status:
acceptedNames:
kind: CephCluster
listKind: CephClusterList
plural: cephclusters
singular: cephcluster
conditions:
- lastTransitionTime: "2020-05-19T18:17:27Z"
message: no conflicts found
reason: NoConflicts
status: "True"
type: NamesAccepted
- lastTransitionTime: "2020-05-19T18:17:27Z"
message: the initial names have been accepted
reason: InitialNamesAccepted
status: "True"
type: Established
- lastTransitionTime: "2020-05-19T18:17:27Z"
message: '[spec.validation.openAPIV3Schema.properties[spec].properties[annotations].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[cephVersion].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[cleanupPolicy].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[dashboard].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[disruptionManagement].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[external].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[mgr].properties[modules].items.type:
Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[mgr].properties[modules].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[mgr].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[mon].properties[volumeClaimTemplate].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[mon].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[monitoring].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[network].properties[selectors].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[network].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[placement].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[rbdMirroring].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[resources].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[config].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[disruptionManagement].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[nodes].items.properties[config].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[nodes].items.properties[devices].items.properties[config].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[nodes].items.properties[devices].items.type:
Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[nodes].items.properties[resources].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[nodes].items.type:
Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[storageClassDeviceSets].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].type:
Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.type:
Required value: must not be empty at the root]'
reason: Violations
status: "True"
type: NonStructuralSchema
storedVersions:
- v1add some info: when i deployed ceph cluster on client node which using config to access k8s cluster, i will meet this issue. But, when i deployed ceph cluster on master node, i will not meet this issue. first, i guess this is an authority issue. But because i used same config no matter master node or client node, so i don't have any clue to find the root cause. my config comes from /etc/kubernetes/admin.conf, i renamed it to config, and then put it in .kube directory. maybe others guy has some clue. thanks.
lss11232
commented
4 years ago [root@k8s-master ~]# kubectl api-resources |grep Ceph cephclients ceph.rook.io true CephClient cephclusters ceph.rook.io true CephCluster cephfilesystems ceph.rook.io true CephFilesystem cephnfses nfs ceph.rook.io true CephNFS cephobjectstores ceph.rook.io true CephObjectStore cephobjectstoreusers rcou,objectuser ceph.rook.io true CephObjectStoreUser
I also encountered a similar problem No CephBlockPool in api-resources
stale[bot]
commented
3 years ago This issue has been automatically marked as stale because it has not had recent activity. It will be closed in a week if no further activity occurs. Thank you for your contributions.
github-actions[bot]
commented
3 years ago This issue has been automatically closed due to inactivity. Please re-open if this still requires investigation.
Ray2310
commented
2 months ago when i deploying a ceph cluster in k8s, i met an error, "error: unable to recognize "cluster.yaml": no matches for kind "CephCluster" in version "ceph.rook.io/v1" i checked by using "kubectl get crd cephclusters.ceph.rook.io" to check whether had created CephCluster kind, it was created. it all looks right, but i can't create ceph cluster after i did common.yaml and operator.yaml
Env: k8s: 1.17.3 Rook: 1.2.7 and 1.3.3
hi， Please tell me if this problem has been solved? I have the same problem
travisn
commented
2 months ago @Ray2310 We could not repro the issue, please provide your install steps to hit this.
Ray2310
commented
2 months ago @Ray2310 We could not repro the issue, please provide your install steps to hit this.
the same issue sir, I cannot provide the steps because I have switched to another plan.
when i deploying a ceph cluster in k8s, i met an error, "error: unable to recognize "cluster.yaml": no matches for kind "CephCluster" in version "ceph.rook.io/v1" i checked by using "kubectl get crd cephclusters.ceph.rook.io" to check whether had created CephCluster kind, it was created. it all looks right, but i can't create ceph cluster after i did common.yaml and operator.yaml
Env: k8s: 1.17.3 Rook: 1.2.7 and 1.3.3