Closed davidkarlsen closed 2 months ago
The core of the problem is that openebs sees the volume as not formatted:
1 mount_linux.go:366] Disk "/dev/datavg/XXX" appears to be unformatted, attempting to format as type: "ext4" with options: [-F -m0 /dev/datavg/XXX]
Did you managed to fix it?
The core of the problem is that openebs sees the volume as not formatted:
1 mount_linux.go:366] Disk "/dev/datavg/XXX" appears to be unformatted, attempting to format as type: "ext4" with options: [-F -m0 /dev/datavg/XXX]
@davidkarlsen It's been long, but did you somehow resolve this for yourself or you still have this issue in some form? The mount message w.r.t unformatted appears to be from kubernetes mount utils code.
@dsharma-dc no longer using openebs myself - but I put in a fix in k8s core to force format xfs and we were OK.
I'm closing this issue now as this isn't a problem anymore as mentioned after the changes in kubernetes mount utils.
What steps did you take and what happened: We had an issue where we replaced the lvmvolume CRDs. Previosly we checked in the CRDs alongside the helm-chart, after the change, we set flux to
this in order to make CRDs be handled with the chart.
This however set deletion-timestamps on the lvmvolume CRs, so in order to avoid the volumes being deleted on reboots, I did
Everything then seems fine, but what we see now, is that if we reboot a node, the volumes comes back empty (!). There are nothing in the logs or events about the volumes being formatted, they just are empty of data.
What can be the cause of this and how can we get out of this serious situation?
What did you expect to happen: Data being preserverd and the volumes mounted as noraml
The output of the following commands will help us better understand what's going on: (Pasting long output into a GitHub gist or other Pastebin is fine.)
kubectl logs -f openebs-lvm-controller-0 -n kube-system -c openebs-lvm-plugin
: https://gist.github.com/davidkarlsen/7adf3b247002f77cfd821d13f688b838kubectl logs -f openebs-lvm-node-[xxxx] -n kube-system -c openebs-lvm-plugin
: https://gist.github.com/davidkarlsen/e99f2a9a2c7d2b407d73ce539cc67aafkubectl get pods -n kube-system
: no podskubectl get lvmvol -A -o yaml
https://gist.github.com/davidkarlsen/6c62b019f5f5624598c618c90bd92805Anything else you would like to add: [Miscellaneous information that will assist in solving the issue.]
Environment:
kubectl version
):/etc/os-release
):