search

LINBIT / linstor-server

High Performance Software-Defined Block Storage for container, cloud and virtualisation. Fully integrated with Docker, Kubernetes, Openstack, Proxmox etc.
https://docs.linbit.com/docs/linstor-guide/
GNU General Public License v3.0
976 stars 76 forks source link

[s3] Unable to create S3 remote due to NotImplemented error #297

Open z9r5 opened 2 years ago

z9r5 commented 2 years ago

linstor 1.18.2

There is the NotImplemented error when creating S3 remote and using non-AWS S3 object storage. Tested with Google Cloud Storage and DigitalOcean Space.

It seems that some functions are being used that are not implemented in DO and Google services but there's no corresponding information in the documentation.

How to reproduce

linstor remote create s3 remote-name <ENDPOINT_URL> bucket-test <REGION> <ACCESS_TOKEN> <SECRET_KEY>

Error report in Google Cloud Storage

============================================================

Application:                        LINBIT�� LINSTOR
Module:                             Controller
Version:                            1.18.2
Build ID:                           26945460e48d2b9e98f6e2163e05b722dd5ff3ca
Build time:                         2022-05-30T09:47:28+00:00
Error time:                         2022-06-30 13:29:52
Node:                               linstor-controller-565c7dc87d-r4whc
Peer:                               RestClient(10.111.0.1; 'PythonLinstor/1.13.1 (API1.0.4): Client 1.13.1')

============================================================

Reported error:
===============

Category:                           RuntimeException
Class name:                         AmazonS3Exception
Class canonical name:               com.amazonaws.services.s3.model.AmazonS3Exception
Generated at:                       Method 'handleErrorResponse', Source file 'AmazonHttpClient.java', Line #1819

Error message:                      A header or query you provided requested a function that is not implemented. (Service: Amazon S3; Status Code: 400; Error Code: NotImplemented; Request ID: null; S3 Extended Request ID: null; Proxy: null)

Error context:
    The remote could not be reached with the given parameters and therefore wasn't created.
    Please check for spelling errors and that you have the correct access-key and secret-key.
    For more information on the error, please check the error-report.

Call backtrace:

    Method                                   Native Class:Line number
    handleErrorResponse                      N      com.amazonaws.http.AmazonHttpClient$RequestExecutor:1819
    handleServiceErrorResponse               N      com.amazonaws.http.AmazonHttpClient$RequestExecutor:1403
    executeOneRequest                        N      com.amazonaws.http.AmazonHttpClient$RequestExecutor:1372
    executeHelper                            N      com.amazonaws.http.AmazonHttpClient$RequestExecutor:1145
    doExecute                                N      com.amazonaws.http.AmazonHttpClient$RequestExecutor:802
    executeWithTimer                         N      com.amazonaws.http.AmazonHttpClient$RequestExecutor:770
    execute                                  N      com.amazonaws.http.AmazonHttpClient$RequestExecutor:744
    access$500                               N      com.amazonaws.http.AmazonHttpClient$RequestExecutor:704
    execute                                  N      com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl:686
    execute                                  N      com.amazonaws.http.AmazonHttpClient:550
    execute                                  N      com.amazonaws.http.AmazonHttpClient:530
    invoke                                   N      com.amazonaws.services.s3.AmazonS3Client:5247
    invoke                                   N      com.amazonaws.services.s3.AmazonS3Client:5194
    invoke                                   N      com.amazonaws.services.s3.AmazonS3Client:5188
    getBucketRequestPayment                  N      com.amazonaws.services.s3.AmazonS3Client:5417
    isRequesterPaysEnabled                   N      com.amazonaws.services.s3.AmazonS3Client:5347
    listObjects                              N      com.linbit.linstor.api.BackupToS3:316
    createS3InTransaction                    N      com.linbit.linstor.core.apicallhandler.controller.CtrlRemoteApiCallHandler:218
    lambda$createS3$0                        N      com.linbit.linstor.core.apicallhandler.controller.CtrlRemoteApiCallHandler:174
    doInScope                                N      com.linbit.linstor.core.apicallhandler.ScopeRunner:147
    lambda$fluxInScope$0                     N      com.linbit.linstor.core.apicallhandler.ScopeRunner:75
    call                                     N      reactor.core.publisher.MonoCallable:91
    trySubscribeScalarMap                    N      reactor.core.publisher.FluxFlatMap:126
    subscribeOrReturn                        N      reactor.core.publisher.MonoFlatMapMany:49
    subscribe                                N      reactor.core.publisher.Flux:8343
    onNext                                   N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:188
    request                                  N      reactor.core.publisher.Operators$ScalarSubscription:2344
    onSubscribe                              N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:134
    subscribe                                N      reactor.core.publisher.MonoCurrentContext:35
    subscribe                                N      reactor.core.publisher.Flux:8357
    onNext                                   N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:188
    request                                  N      reactor.core.publisher.Operators$ScalarSubscription:2344
    onSubscribe                              N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:134
    subscribe                                N      reactor.core.publisher.MonoCurrentContext:35
    subscribe                                N      reactor.core.publisher.Mono:4252
    subscribeWith                            N      reactor.core.publisher.Mono:4363
    subscribe                                N      reactor.core.publisher.Mono:4223
    subscribe                                N      reactor.core.publisher.Mono:4159
    subscribe                                N      reactor.core.publisher.Mono:4131
    doFlux                                   N      com.linbit.linstor.api.rest.v1.RequestHelper:304
    createS3Remote                           N      com.linbit.linstor.api.rest.v1.Remotes:123
    invoke                                   N      jdk.internal.reflect.GeneratedMethodAccessor144:unknown
    invoke                                   N      jdk.internal.reflect.DelegatingMethodAccessorImpl:43
    invoke                                   N      java.lang.reflect.Method:566
    lambda$static$0                          N      org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory:52
    run                                      N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1:124
    invoke                                   N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:167
    doDispatch                               N      org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker:159
    dispatch                                 N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:79
    invoke                                   N      org.glassfish.jersey.server.model.ResourceMethodInvoker:469
    apply                                    N      org.glassfish.jersey.server.model.ResourceMethodInvoker:391
    apply                                    N      org.glassfish.jersey.server.model.ResourceMethodInvoker:80
    run                                      N      org.glassfish.jersey.server.ServerRuntime$1:253
    call                                     N      org.glassfish.jersey.internal.Errors$1:248
    call                                     N      org.glassfish.jersey.internal.Errors$1:244
    process                                  N      org.glassfish.jersey.internal.Errors:292
    process                                  N      org.glassfish.jersey.internal.Errors:274
    process                                  N      org.glassfish.jersey.internal.Errors:244
    runInScope                               N      org.glassfish.jersey.process.internal.RequestScope:265
    process                                  N      org.glassfish.jersey.server.ServerRuntime:232
    handle                                   N      org.glassfish.jersey.server.ApplicationHandler:680
    service                                  N      org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer:356
    run                                      N      org.glassfish.grizzly.http.server.HttpHandler$1:200
    doWork                                   N      org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:569
    run                                      N      org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:549
    run                                      N      java.lang.Thread:829

Error report in DigitalOcean Space

============================================================

Application:                        LINBIT�� LINSTOR
Module:                             Controller
Version:                            1.18.2
Build ID:                           26945460e48d2b9e98f6e2163e05b722dd5ff3ca
Build time:                         2022-05-30T09:47:28+00:00
Error time:                         2022-06-30 12:51:17
Node:                               linstor-controller-565c7dc87d-r4whc
Peer:                               RestClient(10.111.0.1; 'PythonLinstor/1.13.1 (API1.0.4): Client 1.13.1')

============================================================

Reported error:
===============

Category:                           RuntimeException
Class name:                         AmazonS3Exception
Class canonical name:               com.amazonaws.services.s3.model.AmazonS3Exception
Generated at:                       Method 'handleErrorResponse', Source file 'AmazonHttpClient.java', Line #1819

Error message:                      Server does not support the functionality required to fulfill the request. Please see https://developers.digitalocean.com/documentation/spaces/#aws-s3-compatibility (Service: Amazon S3; Status Code: 501; Error Code: NotImplemented; Request ID: null; S3 Extended Request ID: null; Proxy: null)

Error context:
    The remote could not be reached with the given parameters and therefore wasn't created.
    Please check for spelling errors and that you have the correct access-key and secret-key.
    For more information on the error, please check the error-report.

Call backtrace:

    Method                                   Native Class:Line number
    handleErrorResponse                      N      com.amazonaws.http.AmazonHttpClient$RequestExecutor:1819
    handleServiceErrorResponse               N      com.amazonaws.http.AmazonHttpClient$RequestExecutor:1403
    executeOneRequest                        N      com.amazonaws.http.AmazonHttpClient$RequestExecutor:1372
    executeHelper                            N      com.amazonaws.http.AmazonHttpClient$RequestExecutor:1145
    doExecute                                N      com.amazonaws.http.AmazonHttpClient$RequestExecutor:802
    executeWithTimer                         N      com.amazonaws.http.AmazonHttpClient$RequestExecutor:770
    execute                                  N      com.amazonaws.http.AmazonHttpClient$RequestExecutor:744
    access$500                               N      com.amazonaws.http.AmazonHttpClient$RequestExecutor:704
    execute                                  N      com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl:686
    execute                                  N      com.amazonaws.http.AmazonHttpClient:550
    execute                                  N      com.amazonaws.http.AmazonHttpClient:530
    invoke                                   N      com.amazonaws.services.s3.AmazonS3Client:5247
    invoke                                   N      com.amazonaws.services.s3.AmazonS3Client:5194
    invoke                                   N      com.amazonaws.services.s3.AmazonS3Client:5188
    getBucketRequestPayment                  N      com.amazonaws.services.s3.AmazonS3Client:5417
    isRequesterPaysEnabled                   N      com.amazonaws.services.s3.AmazonS3Client:5347
    listObjects                              N      com.linbit.linstor.api.BackupToS3:316
    createS3InTransaction                    N      com.linbit.linstor.core.apicallhandler.controller.CtrlRemoteApiCallHandler:218
    lambda$createS3$0                        N      com.linbit.linstor.core.apicallhandler.controller.CtrlRemoteApiCallHandler:174
    doInScope                                N      com.linbit.linstor.core.apicallhandler.ScopeRunner:147
    lambda$fluxInScope$0                     N      com.linbit.linstor.core.apicallhandler.ScopeRunner:75
    call                                     N      reactor.core.publisher.MonoCallable:91
    trySubscribeScalarMap                    N      reactor.core.publisher.FluxFlatMap:126
    subscribeOrReturn                        N      reactor.core.publisher.MonoFlatMapMany:49
    subscribe                                N      reactor.core.publisher.Flux:8343
    onNext                                   N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:188
    request                                  N      reactor.core.publisher.Operators$ScalarSubscription:2344
    onSubscribe                              N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:134
    subscribe                                N      reactor.core.publisher.MonoCurrentContext:35
    subscribe                                N      reactor.core.publisher.Flux:8357
    onNext                                   N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:188
    request                                  N      reactor.core.publisher.Operators$ScalarSubscription:2344
    onSubscribe                              N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:134
    subscribe                                N      reactor.core.publisher.MonoCurrentContext:35
    subscribe                                N      reactor.core.publisher.Mono:4252
    subscribeWith                            N      reactor.core.publisher.Mono:4363
    subscribe                                N      reactor.core.publisher.Mono:4223
    subscribe                                N      reactor.core.publisher.Mono:4159
    subscribe                                N      reactor.core.publisher.Mono:4131
    doFlux                                   N      com.linbit.linstor.api.rest.v1.RequestHelper:304
    createS3Remote                           N      com.linbit.linstor.api.rest.v1.Remotes:123
    invoke                                   N      jdk.internal.reflect.GeneratedMethodAccessor144:unknown
    invoke                                   N      jdk.internal.reflect.DelegatingMethodAccessorImpl:43
    invoke                                   N      java.lang.reflect.Method:566
    lambda$static$0                          N      org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory:52
    run                                      N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1:124
    invoke                                   N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:167
    doDispatch                               N      org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker:159
    dispatch                                 N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:79
    invoke                                   N      org.glassfish.jersey.server.model.ResourceMethodInvoker:469
    apply                                    N      org.glassfish.jersey.server.model.ResourceMethodInvoker:391
    apply                                    N      org.glassfish.jersey.server.model.ResourceMethodInvoker:80
    run                                      N      org.glassfish.jersey.server.ServerRuntime$1:253
    call                                     N      org.glassfish.jersey.internal.Errors$1:248
    call                                     N      org.glassfish.jersey.internal.Errors$1:244
    process                                  N      org.glassfish.jersey.internal.Errors:292
    process                                  N      org.glassfish.jersey.internal.Errors:274
    process                                  N      org.glassfish.jersey.internal.Errors:244
    runInScope                               N      org.glassfish.jersey.process.internal.RequestScope:265
    process                                  N      org.glassfish.jersey.server.ServerRuntime:232
    handle                                   N      org.glassfish.jersey.server.ApplicationHandler:680
    service                                  N      org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer:356
    run                                      N      org.glassfish.grizzly.http.server.HttpHandler$1:200
    doWork                                   N      org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:569
    run                                      N      org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:549
    run                                      N      java.lang.Thread:829
z9r5 commented 2 years ago

I forgot to add that the --use-path-style option doesn't affect the result.