search

apache / cloudstack

Apache CloudStack is an opensource Infrastructure as a Service (IaaS) cloud computing platform
https://cloudstack.apache.org/
Apache License 2.0
2.11k stars 1.11k forks source link

Cloudstack doesn't check availability of Secondary Storage #9730

Open top-secrett opened 1 month ago

top-secrett commented 1 month ago
ISSUE TYPE
CLOUDSTACK VERSION
4.19.1.0
SUMMARY

I tried to delete Volume Snapshots. Snapshots have been deleted successfully in Cloudstack. but they haven't been deleted on secondary storage. I found logs with error For some reason I've got a secondary storage unavailable from SSMV

root@s-5288-VM:~# ls -la /mnt/SecStorage/440acca8-622d-3dfb-825d-b3e62d7bba87
ls: cannot access '/mnt/SecStorage/440acca8-622d-3dfb-825d-b3e62d7bba87': Stale file handle 
2024-09-25 12:31:45,271 DEBUG [c.c.a.t.Request] (AgentManager-Handler-1:null) (logid:) Seq 77364-5150147648874893418: Processing:  { Ans: , MgmtId: 108597816265214, via: 77364, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":"false","details":"com.clouded to get root directory from secondary storage URL [nfs://10.69.105.254/var/NFS], using NFS version [null], due to [Unable to create local folder for: /mnt/SecStorage/440acca8-622d-3dfb-825d-b3e62d7bba87 in order to mount nfs://10.69.105.254/var/NFS].
2024-09-25 12:31:45,272 DEBUG [o.a.c.s.s.SnapshotServiceImpl] (API-Job-Executor-91:ctx-b803ea6d job-72072 ctx-d8dc0101) (logid:ea5f9ca4) Failed to delete snapshot [49ecc278-7d1c-4d5a-88b8-35eab0127730] due to: [com.cloud.utils.exception.CloudRuntimeException: Failede URL [nfs://10.69.105.254/var/NFS], using NFS version [null], due to [Unable to create local folder for: /mnt/SecStorage/440acca8-622d-3dfb-825d-b3e62d7bba87 in order to mount nfs://10.69.105.254/var/NFS].

I fixed the secondary storage and now new spanshots are deleted successfully, but old ones are stiil present

Expected result

CloudStack checks the availability of the secondary storage and reports when files can not be deleted

Actual result

CloudStack ignores the availability of the storage and silently fails when deletion is not possible.

DaanHoogland commented 1 month ago

@top-secrett can you shed light on what was the problem with the secstor? That will make it easier to create a reproduction scheme and a validation test for a fix when created.

top-secrett commented 1 month ago

@DaanHoogland I stopped secstor VM to increase disk size. I think problem started after that