search

versity / versitygw

versity s3 gateway
https://www.versity.com/products/versitygw/
Apache License 2.0
202 stars 25 forks source link

[Bug] - bucket-list with azure backend error #948

Open TheSpiriter opened 1 week ago

TheSpiriter commented 1 week ago

Describe the bug Listing the buckets via the internal API call does not work if Azure is used as the backend.

To Reproduce

  1. Create a config file with the Azure Blob details /etc/versitygw.d/test.conf 
    VGW_BACKEND=azure
VGW_PORT=:7070
ROOT_ACCESS_KEY_ID=admin
ROOT_SECRET_ACCESS_KEY=test
AZ_ACCOUNT_NAME=STORAGE-ACCOUNT
AZ_ACCESS_KEY=SECRET-KEY
VGW_METRICS_STATSD_SERVERS=127.0.0.1:8125
VGW_HEALTH=/health
VGW_ADMIN_PORT=:7071
VGW_IAM_DIR=/srv/versitygw/iam/STORAGE-ACCOUNT
  2. systemctl enable versitygw@test
systemctl start versitygw@test

    Only works if the file /usr/lib/systemd/system/versitygw@.service is adapted as follows.

    ExecStart=/bin/bash -c 'if [[ ! ("${VGW_BACKEND}" == "posix"|| "${VGW_BACKEND}" == "azure" || "${VGW_BACKEND}" == "scoutfs" || "${VGW_BACKEND}" == "s3") ]]; then echo "VGW_BACKEND environment variable not set to one of posix, scoutfs, or s3"; exit 1; fi && exec /usr/bin/versitygw "$VGW_BACKEND" "$VGW_BACKEND_ARG"'
  3. versitygw admin --access admin --secret test --endpoint-url http://127.0.0.1:7071 list-buckets

Error: 2024/11/15 17:30:07 api error InternalError: We encountered an internal error, please try again.

Unfortunately, I can't manage to get more log data there

Expected behavior That I get a list of the buckets so that I can then assign them an individual ACL. So that only the users who have been assigned these buckets can see them.

Server Version Version : 1.0.8 Build : 45ad3843ef6ee8a325524852e36ed48e45a48721 BuildTime: 2024-10-28T15:42:50Z

Linux ubuntu24 6.8.0-47-generic #47-Ubuntu SMP PREEMPT_DYNAMIC Fri Sep 27 21:40:26 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Additional context For testing use the WinSCP v6.3.5

benmcclelland commented 1 week ago

We have several test cases that run list-buckets against azurite:

RUN  ListBuckets_as_user
PASS ListBuckets_as_user
RUN  ListBuckets_as_admin
PASS ListBuckets_as_admin
RUN  ListBuckets_with_prefix
PASS ListBuckets_with_prefix
RUN  ListBuckets_invalid_max_buckets
PASS ListBuckets_invalid_max_buckets
RUN  ListBuckets_truncated
PASS ListBuckets_truncated
RUN  ListBuckets_success
PASS ListBuckets_success

And I'm adding another to verify the case where no buckets exist:

RUN  ListBuckets_empty_success
PASS ListBuckets_empty_success

These are all passing fine. Is there any more info you can provide about the failure case here? Maybe the output of the server (gateway) side? This should be going to stdout/stderr for the systemd service.

I will work on adding official azure support to systemd service.

TheSpiriter commented 1 week ago

Thank you for your answer. Yes, the Azure endpoint works perfectly. Also via the WinSCP or the azuritegw. List, up- and download works.

However, I am trying to call a ‘list-bucket’ via the versitygw admin. I think the versitygw admin is processing the response incorrectly.

I have activated a log on the Azure blob and the list-bucket requests are different from the client and the versitygw admin. However, both requests are answered with code 200.

TimeGenerated [UTC] Client AccountName Location Protocol OperationName AuthenticationType StatusCode StatusText DurationMs ServerLatencyMs Uri UserAgentHeader Etag ServiceType ObjectKey RequestHeaderSize ResponseHeaderSize ResponseBodySize Category TlsVersion MetricResponseType SourceAccessTier SourceSystem Type
20.11.2024 14:44 versitygw admin azuretestblob switzerlandnorth HTTPS ListContainers AccountKey 200 Success 1 1 https://azuretestblob.blob.core.windows.net:443/?comp=list "azsdk-go-azblob/v1.4.1 (go1.23.2 linux)" blob /azuretestblob 336 152 1252 StorageRead TLS 1.3 Success Invalid Azure StorageBlobLogs
20.11.2024 14:47 WinSCP Client azuretestblob switzerlandnorth HTTPS ListContainers AccountKey 200 Success 4 4 https://azuretestblob.blob.core.windows.net:443/?comp=list&include=metadata "azsdk-go-azblob/v1.4.1 (go1.23.2 linux)" blob /azuretestblob 353 152 1432 StorageRead TLS 1.3 Success Invalid Azure StorageBlobLogs
benmcclelland commented 1 day ago

Oh! this is with the admin command. I missed that before. Yes, I can reproduce the error on azurite now. Looks like the issue requires buckets to exist:

% ./versitygw admin -a user -s pass -r us-east-1 --er http://127.0.0.1:7070 list-buckets
Bucket   Owner
-------  ----

% aws s3 mb s3://testbucket1
make_bucket: testbucket1
% aws s3 mb s3://testbucket2
make_bucket: testbucket2

% ./versitygw admin -a user -s pass -r us-east-1 --er http://127.0.0.1:7070 list-buckets
2024/11/25 10:11:08 api error InternalError: We encountered an internal error, please try again.

and the server side logs:

azurite-1    | 172.20.0.3 - - [25/Nov/2024:18:11:08 +0000] "GET /devstoreaccount1?comp=list HTTP/1.1" 200 -
azuritegw-1  | 2024/11/25 18:11:08 stdout: 18:11:07 | 500 |    9.425666ms | 172.20.0.1 | PATCH | /list-buckets | - |