search

LINBIT / linstor-server

High Performance Software-Defined Block Storage for container, cloud and virtualisation. Fully integrated with Docker, Kubernetes, Openstack, Proxmox etc.
https://docs.linbit.com/docs/linstor-guide/
GNU General Public License v3.0
984 stars 76 forks source link

Failed to create snapshot #369

Closed boedy closed 1 year ago

boedy commented 1 year ago

Failed to check and update snapshot content: failed to take snapshot of the volume pvc-a048fb80-12e7-4aa7-b5a6-a7befecee4f7: "rpc error: code = Internal desc = failed to create snapshot: failed to create snapshot: Message: 'Storage driver 'FILE_THIN' does not support snapshots.'; Details: 'Used for storage pool 'data-dir' on 'h-nbg-51b6'.\nResource: pvc-a048fb80-12e7-4aa7-b5a6-a7befecee4f7, Snapshot: snapshot-64dc1795-9198-4ea6-a727-426a8048d8cb'; Reports: '[650E5FE7-00000-001047]'"

error report:

ERROR REPORT 650E5FE7-00000-001047

============================================================

Application:                        LINBIT�� LINSTOR
Module:                             Controller
Version:                            1.24.1
Build ID:                           f7d71e7c5416c84d6cdb05f9b490d9b1e81622e2
Build time:                         2023-08-10T09:13:30+00:00
Error time:                         2023-09-26 10:47:29
Node:                               linstor-controller-5f987d7ff5-mr6n2
Peer:                               RestClient(192.168.179.0; 'linstor-csi/v1.2.2-81368b0dd589ca92d5a96d4e46fb5a2341f386c7')

============================================================

Reported error:
===============

Category:                           RuntimeException
Class name:                         ApiRcException
Class canonical name:               com.linbit.linstor.core.apicallhandler.response.ApiRcException
Generated at:                       Method 'ensureDriversSupportSnapshots', Source file 'CtrlSnapshotCrtHelper.java', Line #352

Error message:                      Storage driver 'FILE_THIN' does not support snapshots.

Error context:
    Storage driver 'FILE_THIN' does not support snapshots.

Asynchronous stage backtrace:

    Error has been observed at the following site(s):
        *__checkpoint ? Create (multi) snapshot
    Original Stack Trace:

Call backtrace:

    Method                                   Native Class:Line number
    ensureDriversSupportSnapshots            N      com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotCrtHelper:352

Suppressed exception 1 of 1:
===============
Category:                           RuntimeException
Class name:                         OnAssemblyException
Class canonical name:               reactor.core.publisher.FluxOnAssembly.OnAssemblyException
Generated at:                       Method 'ensureDriversSupportSnapshots', Source file 'CtrlSnapshotCrtHelper.java', Line #352

Error message:
Error has been observed at the following site(s):
    *__checkpoint ��� Create (multi) snapshot
Original Stack Trace:

Error context:
    Storage driver 'FILE_THIN' does not support snapshots.

Call backtrace:

    Method                                   Native Class:Line number
    ensureDriversSupportSnapshots            N      com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotCrtHelper:352
    ensureSnapshotsViable                    N      com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotCrtHelper:315
    createSnapshots                          N      com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotCrtHelper:123
    createSnapshotInTransaction              N      com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotCrtApiCallHandler:176
    lambda$createSnapshot$0                  N      com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotCrtApiCallHandler:160
    doInScope                                N      com.linbit.linstor.core.apicallhandler.ScopeRunner:149
    lambda$fluxInScope$0                     N      com.linbit.linstor.core.apicallhandler.ScopeRunner:76
    call                                     N      reactor.core.publisher.MonoCallable:72
    trySubscribeScalarMap                    N      reactor.core.publisher.FluxFlatMap:127
    subscribeOrReturn                        N      reactor.core.publisher.MonoFlatMapMany:49
    subscribe                                N      reactor.core.publisher.Flux:8759
    onNext                                   N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:195
    request                                  N      reactor.core.publisher.Operators$ScalarSubscription:2545
    onSubscribe                              N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:141
    subscribe                                N      reactor.core.publisher.MonoJust:55
    subscribe                                N      reactor.core.publisher.MonoDeferContextual:55
    subscribe                                N      reactor.core.publisher.Flux:8773
    onNext                                   N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:195
    request                                  N      reactor.core.publisher.Operators$ScalarSubscription:2545
    onSubscribe                              N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:141
    subscribe                                N      reactor.core.publisher.MonoJust:55
    subscribe                                N      reactor.core.publisher.MonoDeferContextual:55
    subscribe                                N      reactor.core.publisher.Mono:4495
    subscribeWith                            N      reactor.core.publisher.Mono:4561
    subscribe                                N      reactor.core.publisher.Mono:4462
    subscribe                                N      reactor.core.publisher.Mono:4398
    subscribe                                N      reactor.core.publisher.Mono:4370
    doFlux                                   N      com.linbit.linstor.api.rest.v1.RequestHelper:303
    createSnapshot                           N      com.linbit.linstor.api.rest.v1.Snapshots:145
    invoke                                   N      jdk.internal.reflect.GeneratedMethodAccessor206:unknown
    invoke                                   N      jdk.internal.reflect.DelegatingMethodAccessorImpl:43
    invoke                                   N      java.lang.reflect.Method:566
    lambda$static$0                          N      org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory:52
    run                                      N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1:146
    invoke                                   N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:189
    doDispatch                               N      org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker:159
    dispatch                                 N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:93
    invoke                                   N      org.glassfish.jersey.server.model.ResourceMethodInvoker:478
    apply                                    N      org.glassfish.jersey.server.model.ResourceMethodInvoker:400
    apply                                    N      org.glassfish.jersey.server.model.ResourceMethodInvoker:81
    run                                      N      org.glassfish.jersey.server.ServerRuntime$1:256
    call                                     N      org.glassfish.jersey.internal.Errors$1:248
    call                                     N      org.glassfish.jersey.internal.Errors$1:244
    process                                  N      org.glassfish.jersey.internal.Errors:292
    process                                  N      org.glassfish.jersey.internal.Errors:274
    process                                  N      org.glassfish.jersey.internal.Errors:244
    runInScope                               N      org.glassfish.jersey.process.internal.RequestScope:265
    process                                  N      org.glassfish.jersey.server.ServerRuntime:235
    handle                                   N      org.glassfish.jersey.server.ApplicationHandler:684
    service                                  N      org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer:356
    run                                      N      org.glassfish.grizzly.http.server.HttpHandler$1:190
    doWork                                   N      org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:535
    run                                      N      org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:515
    run                                      N      java.lang.Thread:829

END OF ERROR REPORT.
boedy commented 1 year ago

I was using a ext4 filesystem. This doesn't seem to be supported. After moving the resource to a new satellite with XFS filesystem I was able to perform a snapshot.

Shout-out to @WanzenBug for the help:

Seems LINSTOR is (sensibly) checking if the FS supports cp --reflink=always. That is supported by XFS and btrfs, but I think still not supported by ext4 for example