search

SynologyOpenSource / synology-csi

Apache License 2.0
499 stars 107 forks source link

Corrupted filesystem #22

Open tyzbit opened 2 years ago

tyzbit commented 2 years ago

When first provisioning, the CSI driver fills up the LUN, which takes some time. I suspect because of this, I experienced filesystem errors on a mounted device and lost data.

image
dcardellino commented 2 years ago

@tyzbit I had a similar issue. I resolved it using the thin provision parameter:

parameters:
  <other parameters go here>
  type: thin
tyzbit commented 2 years ago

@dcardellino That worked great, the space used is now accurate. I suggest this option be added as a default parameter to the StorageClass.

EDIT: It appears these LUNs are not snapshottable though. Is there a workaround?

chihyuwu commented 2 years ago

Hi @tyzbit, Try to use BLUN or ADV instead of thin. If you still get filesystem errors after that, please let us know.

There're 4 types of LUN that the CSI driver can create: BLUN, BLUN_THICK, ADV and FILE, and only BLUN and ADV are snapshottable. In general, you don't have to assign the type parameter by yourself.

tyzbit commented 2 years ago

@chihyuwu Thanks for that info, I tried BLUN and ADV and both created thin LUNs and the space started filling up like before, so I suspect there will continue to be issues. I also noticed ADV cannot be snapshotted.

chihyuwu commented 2 years ago

@tyzbit Filling up LUNs is an expected behavior and it won't corrupt your filesystem, https://github.com/SynologyOpenSource/synology-csi/issues/25#issuecomment-1049710748 .

I experienced filesystem errors on a mounted device and lost data.

There may be some other reasons for it. Can you give more details or maybe a screenshot?

tyzbit commented 2 years ago

I created a PVC without the type field in the StorageClass parameters and filled it with data using a Job. Then when I tried to remount it, I get these errors:

Events:
  Type     Reason                  Age               From                     Message
  ----     ------                  ----              ----                     -------
  Normal   Scheduled               49s               default-scheduler        Successfully assigned media/migrate-pv-stash-previews-nzkpp to weatherman
  Normal   SuccessfulAttachVolume  48s               attachdetach-controller  AttachVolume.Attach succeeded for volume "pvc-845e6b97-93ad-4778-ad89-e49b8d470858"
  Warning  FailedMount             0s (x7 over 38s)  kubelet                  MountVolume.MountDevice failed for volume "pvc-845e6b97-93ad-4778-ad89-e49b8d470858" : rpc error: code = Internal desc = mount failed: exit status 32
Mounting command: mount
Mounting arguments: -t btrfs -o rw,defaults /dev/disk/by-path/ip-192.168.1.59:3260-iscsi-iqn.2000-01.com.synology:nas.pvc-845e6b97-93ad-4778-ad89-e49b8d470858-lun-1 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-845e6b97-93ad-4778-ad89-e49b8d470858/globalmount
Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-845e6b97-93ad-4778-ad89-e49b8d470858/globalmount: wrong fs type, bad option, bad superblock on /dev/sdf, missing codepage or helper program, or other error.