search

LINBIT / linstor-server

High Performance Software-Defined Block Storage for container, cloud and virtualisation. Fully integrated with Docker, Kubernetes, Openstack, Proxmox etc.
https://docs.linbit.com/docs/linstor-guide/
GNU General Public License v3.0
977 stars 76 forks source link

Unable to parse internal value of internal property SequenceNumber #320

Open blampe opened 2 years ago

blampe commented 2 years ago

More adventures with snapshotting.

============================================================

Application:                        LINBIT�� LINSTOR
Module:                             Controller
Version:                            1.20.0-rc.1
Build ID:                           095b9bef67d46f217ee394e0262c4e96baef0c45
Build time:                         2022-09-20T12:44:59+00:00
Error time:                         2022-09-24 18:47:16
Node:                               piraeus-op-cs-controller-f8fdf977f-cxx28
Peer:                               RestClient(10.42.1.250; 'linstor-csi/v0.20.0-d514e41db7cdcb580769cc69f1c1ef2b8a5def5d')

============================================================

Reported error:
===============

Category:                           Error
Class name:                         ImplementationError
Class canonical name:               com.linbit.ImplementationError
Generated at:                       Method 'maxSequenceNumber', Source file 'SnapshotDefinitionControllerFactory.java', Line #127

Error message:                      Unable to parse internal value of internal property SequenceNumber

Error context:
    Creation of snapshot 'snapshot-11fa4374-7b92-4d01-afd9-872e6210add1' of resource 'pvc-23f70722-ab5d-418d-8742-59b3ce0a6ef7' failed due to an implementation error.

Asynchronous stage backtrace:

    Error has been observed at the following site(s):
        |_ checkpoint ? Create snapshot
    Stack trace:

Call backtrace:

    Method                                   Native Class:Line number
    maxSequenceNumber                        N      com.linbit.linstor.core.objects.SnapshotDefinitionControllerFactory:127

Suppressed exception 1 of 1:
===============
Category:                           RuntimeException
Class name:                         OnAssemblyException
Class canonical name:               reactor.core.publisher.FluxOnAssembly.OnAssemblyException
Generated at:                       Method 'maxSequenceNumber', Source file 'SnapshotDefinitionControllerFactory.java', Line #127

Error message:
Error has been observed at the following site(s):
    |_ checkpoint ��� Create snapshot
Stack trace:

Error context:
    Creation of snapshot 'snapshot-11fa4374-7b92-4d01-afd9-872e6210add1' of resource 'pvc-23f70722-ab5d-418d-8742-59b3ce0a6ef7' failed due to an implementation error.

Call backtrace:

    Method                                   Native Class:Line number
    maxSequenceNumber                        N      com.linbit.linstor.core.objects.SnapshotDefinitionControllerFactory:127
    create                                   N      com.linbit.linstor.core.objects.SnapshotDefinitionControllerFactory:89
    createSnapshotDfnData                    N      com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotCrtHelper:384
    createSnapshots                          N      com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotCrtHelper:117
    createSnapshotInTransaction              N      com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotCrtApiCallHandler:147
    lambda$createSnapshot$0                  N      com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotCrtApiCallHandler:134
    doInScope                                N      com.linbit.linstor.core.apicallhandler.ScopeRunner:150
    lambda$fluxInScope$0                     N      com.linbit.linstor.core.apicallhandler.ScopeRunner:76
    call                                     N      reactor.core.publisher.MonoCallable:91
    trySubscribeScalarMap                    N      reactor.core.publisher.FluxFlatMap:126
    subscribeOrReturn                        N      reactor.core.publisher.MonoFlatMapMany:49
    subscribe                                N      reactor.core.publisher.Flux:8343
    onNext                                   N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:188
    request                                  N      reactor.core.publisher.Operators$ScalarSubscription:2344
    onSubscribe                              N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:134
    subscribe                                N      reactor.core.publisher.MonoCurrentContext:35
    subscribe                                N      reactor.core.publisher.Flux:8357
    onNext                                   N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:188
    request                                  N      reactor.core.publisher.Operators$ScalarSubscription:2344
    onSubscribe                              N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:134
    subscribe                                N      reactor.core.publisher.MonoCurrentContext:35
    subscribe                                N      reactor.core.publisher.Mono:4252
    subscribeWith                            N      reactor.core.publisher.Mono:4363
    subscribe                                N      reactor.core.publisher.Mono:4223
    subscribe                                N      reactor.core.publisher.Mono:4159
    subscribe                                N      reactor.core.publisher.Mono:4131
    doFlux                                   N      com.linbit.linstor.api.rest.v1.RequestHelper:304
    createSnapshot                           N      com.linbit.linstor.api.rest.v1.Snapshots:145
    invoke                                   N      jdk.internal.reflect.GeneratedMethodAccessor198:unknown
    invoke                                   N      jdk.internal.reflect.DelegatingMethodAccessorImpl:43
    invoke                                   N      java.lang.reflect.Method:566
    lambda$static$0                          N      org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory:52
    run                                      N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1:124
    invoke                                   N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:167
    doDispatch                               N      org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker:159
    dispatch                                 N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:79
    invoke                                   N      org.glassfish.jersey.server.model.ResourceMethodInvoker:469
    apply                                    N      org.glassfish.jersey.server.model.ResourceMethodInvoker:391
    apply                                    N      org.glassfish.jersey.server.model.ResourceMethodInvoker:80
    run                                      N      org.glassfish.jersey.server.ServerRuntime$1:253
    call                                     N      org.glassfish.jersey.internal.Errors$1:248
    call                                     N      org.glassfish.jersey.internal.Errors$1:244
    process                                  N      org.glassfish.jersey.internal.Errors:292
    process                                  N      org.glassfish.jersey.internal.Errors:274
    process                                  N      org.glassfish.jersey.internal.Errors:244
    runInScope                               N      org.glassfish.jersey.process.internal.RequestScope:265
    process                                  N      org.glassfish.jersey.server.ServerRuntime:232
    handle                                   N      org.glassfish.jersey.server.ApplicationHandler:680
    service                                  N      org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer:356
    run                                      N      org.glassfish.grizzly.http.server.HttpHandler$1:200
    doWork                                   N      org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:569
    run                                      N      org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:549
    run                                      N      java.lang.Thread:829

Caused by:
==========

Category:                           RuntimeException
Class name:                         NumberFormatException
Class canonical name:               java.lang.NumberFormatException
Generated at:                       Method 'parseLong', Source file 'Long.java', Line #655

Error message:                      null

Call backtrace:

    Method                                   Native Class:Line number
    parseLong                                N      java.lang.Long:655
    valueOf                                  N      java.lang.Long:1144
    maxSequenceNumber                        N      com.linbit.linstor.core.objects.SnapshotDefinitionControllerFactory:115
    create                                   N      com.linbit.linstor.core.objects.SnapshotDefinitionControllerFactory:89
    createSnapshotDfnData                    N      com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotCrtHelper:384
    createSnapshots                          N      com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotCrtHelper:117
    createSnapshotInTransaction              N      com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotCrtApiCallHandler:147
    lambda$createSnapshot$0                  N      com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotCrtApiCallHandler:134
    doInScope                                N      com.linbit.linstor.core.apicallhandler.ScopeRunner:150
    lambda$fluxInScope$0                     N      com.linbit.linstor.core.apicallhandler.ScopeRunner:76
    call                                     N      reactor.core.publisher.MonoCallable:91
    trySubscribeScalarMap                    N      reactor.core.publisher.FluxFlatMap:126
    subscribeOrReturn                        N      reactor.core.publisher.MonoFlatMapMany:49
    subscribe                                N      reactor.core.publisher.Flux:8343
    onNext                                   N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:188
    request                                  N      reactor.core.publisher.Operators$ScalarSubscription:2344
    onSubscribe                              N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:134
    subscribe                                N      reactor.core.publisher.MonoCurrentContext:35
    subscribe                                N      reactor.core.publisher.Flux:8357
    onNext                                   N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:188
    request                                  N      reactor.core.publisher.Operators$ScalarSubscription:2344
    onSubscribe                              N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:134
    subscribe                                N      reactor.core.publisher.MonoCurrentContext:35
    subscribe                                N      reactor.core.publisher.Mono:4252
    subscribeWith                            N      reactor.core.publisher.Mono:4363
    subscribe                                N      reactor.core.publisher.Mono:4223
    subscribe                                N      reactor.core.publisher.Mono:4159
    subscribe                                N      reactor.core.publisher.Mono:4131
    doFlux                                   N      com.linbit.linstor.api.rest.v1.RequestHelper:304
    createSnapshot                           N      com.linbit.linstor.api.rest.v1.Snapshots:145
    invoke                                   N      jdk.internal.reflect.GeneratedMethodAccessor198:unknown
    invoke                                   N      jdk.internal.reflect.DelegatingMethodAccessorImpl:43
    invoke                                   N      java.lang.reflect.Method:566
    lambda$static$0                          N      org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory:52
    run                                      N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1:124
    invoke                                   N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:167
    doDispatch                               N      org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker:159
    dispatch                                 N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:79
    invoke                                   N      org.glassfish.jersey.server.model.ResourceMethodInvoker:469
    apply                                    N      org.glassfish.jersey.server.model.ResourceMethodInvoker:391
    apply                                    N      org.glassfish.jersey.server.model.ResourceMethodInvoker:80
    run                                      N      org.glassfish.jersey.server.ServerRuntime$1:253
    call                                     N      org.glassfish.jersey.internal.Errors$1:248
    call                                     N      org.glassfish.jersey.internal.Errors$1:244
    process                                  N      org.glassfish.jersey.internal.Errors:292
    process                                  N      org.glassfish.jersey.internal.Errors:274
    process                                  N      org.glassfish.jersey.internal.Errors:244
    runInScope                               N      org.glassfish.jersey.process.internal.RequestScope:265
    process                                  N      org.glassfish.jersey.server.ServerRuntime:232
    handle                                   N      org.glassfish.jersey.server.ApplicationHandler:680
    service                                  N      org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer:356
    run                                      N      org.glassfish.grizzly.http.server.HttpHandler$1:200
    doWork                                   N      org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:569
    run                                      N      org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:549
    run                                      N      java.lang.Thread:829
ghernadi commented 2 years ago

Apparently one of your SnapshotDefinitions is missing the (internal) property SequenceNumber. I do not see any obvious way to lose that property. Any idea how that could have happened? Any external database-modifications involved here?