Platform: NC (running locally on MacOS M1 arch: linux/arm64)
Actual behavior
When listing buckets and one of the buckets has an invalid schema the list buckets throw an error of InvalidBucketState.
Expected behavior
When listing buckets and one of the buckets has an invalid schema the list of the buckets should appear without the bucket with an invalid schema and print it in the logs.
Steps to reproduce
Create an account using the CLI (shira-1001): sudo node src/cmd/manage_nsfs account add --name shira-1001 --new_buckets_path /tmp/nsfs_root1 --access_key <access-key> --secret_key <secret-key> --uid <uid> --gid <gid>
Note: before creating the account need to give permission to the new_buckets_path: chmod 777 /tmp/nsfs_root1.
Start the NSFS server with: sudo node src/cmd/nsfs --debug 5
Note: before starting the server please add this line: process.env.NOOBAA_LOG_LEVEL = 'nsfs'; in the endpoint.js (before the condition if (process.env.NOOBAA_LOG_LEVEL) {)
Create the alias for S3 service: alias s3-nc-user-1='AWS_ACCESS_KEY_ID=<access-key> AWS_SECRET_ACCESS_KEY=<secret-key> aws --no-verify-ssl --endpoint-url https://localhost:6443'.
Create 2 buckets in the S3 flow (bucket1, bucket2): s3-nc-user-1-s3 s3 mb s3://bucket1, s3-nc-user-1-s3 s3 mb s3://bucket2
List the buckets (should see the 2 buckets): s3-nc-user-1-s3 s3 ls
Edit the JSON schema config of one of the buckets (bucket2), for example, add "a":"a" (additional properties are not allowed): sudo vi /etc/noobaa.conf.d/buckets/bucket2.json
List the buckets - currently you would receive the InvalidBucketState error: s3-nc-user-1-s3 s3 ls
More information - Screenshots / Logs / Other output
S3 output:
An error occurred (InvalidBucketState) when calling the ListBuckets operation: The request is not valid with the current state of the bucket.
NSFS Logs:
Jul-25 11:40:12.941 [nsfs/78985] [WARN] core.sdk.bucketspace_fs:: list_buckets: read_bucket_sdk_config_info of bucket bucket2 got an error RpcError: must NOT have additional properties | {"additionalProperty":"a"}
at Object.validate_bucket_schema (/Users/shiradymnik/SourceCode/noobaa-core/src/manage_nsfs/nsfs_schema_utils.js:72:15)
at BucketSpaceFS.read_bucket_sdk_info (/Users/shiradymnik/SourceCode/noobaa-core/src/sdk/bucketspace_fs.js:135:31)
at async NsfsObjectSDK._load_bucket_namespace (/Users/shiradymnik/SourceCode/noobaa-core/src/sdk/object_sdk.js:283:41)
at async LRUCache._load_item (/Users/shiradymnik/SourceCode/noobaa-core/src/util/lru_cache.js:83:22)
at async LRUCache.get_with_cache (/Users/shiradymnik/SourceCode/noobaa-core/src/util/lru_cache.js:72:13)
at async NsfsObjectSDK.read_bucket_sdk_config_info (/Users/shiradymnik/SourceCode/noobaa-core/src/sdk/object_sdk.js:180:28)
at async /Users/shiradymnik/SourceCode/noobaa-core/src/sdk/bucketspace_fs.js:236:26
at async Semaphore.surround (/Users/shiradymnik/SourceCode/noobaa-core/src/util/semaphore.js:71:84)
at async Promise.all (index 2)
at async BucketSpaceFS.list_buckets (/Users/shiradymnik/SourceCode/noobaa-core/src/sdk/bucketspace_fs.js:229:25) {
rpc_code: 'INVALID_BUCKET_STATE'
}
Jul-25 11:40:12.942 [nsfs/78985] [ERROR] core.endpoint.s3.s3_rest:: S3 ERROR <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidBucketState</Code><Message>The request is not valid with the current state of the bucket.</Message><Resource>/</Resource><RequestId>lz10wk1r-203n9-66m</RequestId></Error> GET / {"host":"localhost:6443","accept-encoding":"identity","user-agent":"aws-cli/2.17.11 md/awscrt#0.20.11 ua/2.0 os/macos#23.4.0 md/arch#arm64 lang/python#3.11.9 md/pyimpl#CPython cfg/retry-mode#standard md/installer#source md/prompt#off md/command#s3.ls","x-amz-date":"20240725T084012Z","x-amz-content-sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","authorization":"AWS4-HMAC-SHA256 Credential=Dwertyuiopasdfg11001/20240725/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=0acfa6c612fffcc8090bf1bdcc388a99dabc4ceda1cfb482cd3dc733de871434"} Error: must NOT have additional properties | {"additionalProperty":"a"}
at Object.validate_bucket_schema (/Users/shiradymnik/SourceCode/noobaa-core/src/manage_nsfs/nsfs_schema_utils.js:72:15)
at BucketSpaceFS.read_bucket_sdk_info (/Users/shiradymnik/SourceCode/noobaa-core/src/sdk/bucketspace_fs.js:135:31)
at async NsfsObjectSDK._load_bucket_namespace (/Users/shiradymnik/SourceCode/noobaa-core/src/sdk/object_sdk.js:283:41)
at async LRUCache._load_item (/Users/shiradymnik/SourceCode/noobaa-core/src/util/lru_cache.js:83:22)
at async LRUCache.get_with_cache (/Users/shiradymnik/SourceCode/noobaa-core/src/util/lru_cache.js:72:13)
at async NsfsObjectSDK.read_bucket_sdk_config_info (/Users/shiradymnik/SourceCode/noobaa-core/src/sdk/object_sdk.js:180:28)
at async /Users/shiradymnik/SourceCode/noobaa-core/src/sdk/bucketspace_fs.js:236:26
at async Semaphore.surround (/Users/shiradymnik/SourceCode/noobaa-core/src/util/semaphore.js:71:84)
at async Promise.all (index 2)
at async BucketSpaceFS.list_buckets (/Users/shiradymnik/SourceCode/noobaa-core/src/sdk/bucketspace_fs.js:229:25)
Environment info
Actual behavior
InvalidBucketState
.Expected behavior
Steps to reproduce
sudo node src/cmd/manage_nsfs account add --name shira-1001 --new_buckets_path /tmp/nsfs_root1 --access_key <access-key> --secret_key <secret-key> --uid <uid> --gid <gid>
Note: before creating the account need to give permission to thenew_buckets_path
:chmod 777 /tmp/nsfs_root1
.sudo node src/cmd/nsfs --debug 5
Note: before starting the server please add this line:process.env.NOOBAA_LOG_LEVEL = 'nsfs';
in the endpoint.js (before the conditionif (process.env.NOOBAA_LOG_LEVEL) {
)alias s3-nc-user-1='AWS_ACCESS_KEY_ID=<access-key> AWS_SECRET_ACCESS_KEY=<secret-key> aws --no-verify-ssl --endpoint-url https://localhost:6443'
.s3-nc-user-1-s3 s3 mb s3://bucket1
,s3-nc-user-1-s3 s3 mb s3://bucket2
s3-nc-user-1-s3 s3 ls
"a":"a"
(additional properties are not allowed):sudo vi /etc/noobaa.conf.d/buckets/bucket2.json
InvalidBucketState
error:s3-nc-user-1-s3 s3 ls
More information - Screenshots / Logs / Other output
S3 output:
An error occurred (InvalidBucketState) when calling the ListBuckets operation: The request is not valid with the current state of the bucket.
NSFS Logs: