hi all
I have set up a kubernetes cluster with the following specs:
kubeadm version 1.18.2
8 nodes with ubuntu 18.04 LTS
I have edited the kube-templates scripts for Deployment apiVersion to be apps/v1 and Service apiVersion to v1 and also added the spec.selector: to wokrs on kubernetes version 1.18.2
the glusterfs-server glusterfs-common and glusterfs-client packags are installed on the nodes
for glusterfs storage I have followed the instuctions, edit the topology file and add the needed firewall rules
but it stock at the Error waiting for job 'heketi-storage-copy-job' to complete. stage
here are the heketi-storage-copy-job events:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled <unknown> default-scheduler Successfully assigned glusterfs/heketi-storage-copy-job-ftqps to server-xxx
Warning FailedMount 19m kubelet, server-xxx MountVolume.SetUp failed for volume "heketi-storage" : mount failed: mount failed: exit status 1
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/2aca16e0-9080-440e-8340-0ab1ad8ffbff/volumes/kubernetes.io~glusterfs/heketi-stor
age --scope -- mount -t glusterfs -o auto_unmount,backup-volfile-servers=10.0.0.1:10.0.0.2:10.0.0.3,log-file=/var/lib/kubelet/plugins/kubernetes.io/
glusterfs/heketi-storage/heketi-storage-copy-job-ftqps-glusterfs.log,log-level=ERROR 10.0.0.2:heketidbstorage /var/lib/kubelet/pods/2aca16e0-9080-440e-8340-0ab1a
d8ffbff/volumes/kubernetes.io~glusterfs/heketi-storage
Output: Running scope as unit: run-r7c281513866a4b04b6ca2737d65e886f.scope
Mount failed. Please check the log file for more details.
, the following error information was pulled from the glusterfs log to help diagnose this issue:
[2020-05-17 16:40:22.243907] E [MSGID: 100026] [glusterfsd.c:2403:glusterfs_process_volfp] 0-: failed to construct the graph
[2020-05-17 16:40:22.244187] E [graph.c:1102:glusterfs_graph_destroy] (-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x537) [0x556eaf57fe77] -->/usr/sbin/glusterfs(glusterfs
_process_volfp+0x180) [0x556eaf579ea0] -->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(glusterfs_graph_destroy+0x6b) [0x7f29a9580eab] ) 0-graph: invalid argument: graph
[Invalid argument]
Warning FailedMount 19m kubelet, srvarvan-xxx MountVolume.SetUp failed for volume "heketi-storage" : mount failed: mount failed: exit status 1
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/2aca16e0-9080-440e-8340-0ab1ad8ffbff/volumes/kubernetes.io~glusterfs/heketi-stor
age --scope -- mount -t glusterfs -o auto_unmount,backup-volfile-servers=10.0.0.1:10.0.0.2:10.0.0.3,log-file=/var/lib/kubelet/plugins/kubernetes.io/
glusterfs/heketi-storage/heketi-storage-copy-job-ftqps-glusterfs.log,log-level=ERROR 10.0.0.2:heketidbstorage /var/lib/kubelet/pods/2aca16e0-9080-440e-8340-0ab1a
d8ffbff/volumes/kubernetes.io~glusterfs/heketi-storage
Output: Running scope as unit: run-rd71a89eb73ad41e4a9c2502f46a6121c.scope
Mount failed. Please check the log file for more details.
, the following error information was pulled from the glusterfs log to help diagnose this issue:
[2020-05-17 16:40:22.933491] E [MSGID: 100026] [glusterfsd.c:2403:glusterfs_process_volfp] 0-: failed to construct the graph
[2020-05-17 16:40:22.933844] E [graph.c:1102:glusterfs_graph_destroy] (-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x537) [0x56267e02ce77] -->/usr/sbin/glusterfs(glusterfs
_process_volfp+0x180) [0x56267e026ea0] -->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(glusterfs_graph_destroy+0x6b) [0x7ff942e3deab] ) 0-graph: invalid argument: graph
[Invalid argument]
Warning FailedMount 19m kubelet, srvarvan-xxx MountVolume.SetUp failed for volume "heketi-storage" : mount failed: mount failed: exit status 1
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/2aca16e0-9080-440e-8340-0ab1ad8ffbff/volumes/kubernetes.io~glusterfs/heketi-stor
age --scope -- mount -t glusterfs -o auto_unmount,backup-volfile-servers=10.0.0.1:10.0.0.2:10.0.0.3,log-file=/var/lib/kubelet/plugins/kubernetes.io/
glusterfs/heketi-storage/heketi-storage-copy-job-ftqps-glusterfs.log,log-level=ERROR 10.0.0.3:heketidbstorage /var/lib/kubelet/pods/2aca16e0-9080-440e-8340-0ab1a
d8ffbff/volumes/kubernetes.io~glusterfs/heketi-storage
Output: Running scope as unit: run-rd71a89eb73ad41e4a9c2502f46a6121c.scope
Mount failed. Please check the log file for more details.
, the following error information was pulled from the glusterfs log to help diagnose this issue:
[2020-05-17 16:40:22.933491] E [MSGID: 100026] [glusterfsd.c:2403:glusterfs_process_volfp] 0-: failed to construct the graph
[2020-05-17 16:40:22.933844] E [graph.c:1102:glusterfs_graph_destroy] (-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x537) [0x56267e02ce77] -->/usr/sbin/glusterfs(glusterfs
_process_volfp+0x180) [0x56267e026ea0] -->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(glusterfs_graph_destroy+0x6b) [0x7ff942e3deab] ) 0-graph: invalid argument: graph
.......
Warning FailedMount 17m kubelet, srvarvan-xxx Unable to attach or mount volumes: unmounted volumes=[heketi-storage], unattached volumes=[heke
ti-storage-secret default-token-55czn heketi-storage]: timed out waiting for the condition
Warning FailedMount 68s (x2 over 12m) kubelet, srvarvan-xxx Unable to attach or mount volumes: unmounted volumes=[heketi-storage], unattached volumes=[heke
ti-storage heketi-storage-secret default-token-55czn]: timed out waiting for the condition
Warning FailedMount 42s (x14 over 16m) kubelet, srvarvan-xxx (combined from similar events): MountVolume.SetUp failed for volume "heketi-storage" : mount fa
iled: mount failed: exit status 1
Mounting command: systemd-run
hi all I have set up a kubernetes cluster with the following specs: kubeadm version 1.18.2 8 nodes with ubuntu 18.04 LTS
I have edited the kube-templates scripts for Deployment apiVersion to be apps/v1 and Service apiVersion to v1 and also added the spec.selector: to wokrs on kubernetes version 1.18.2
the glusterfs-server glusterfs-common and glusterfs-client packags are installed on the nodes
for glusterfs storage I have followed the instuctions, edit the topology file and add the needed firewall rules but it stock at the Error waiting for job 'heketi-storage-copy-job' to complete. stage
here are the heketi-storage-copy-job events:
what am I doing wrong ?