NSFS | NC | List Buckets Fails With `InvalidBucketState` In Case a Bucket Has Invalid Schema Config File

[shirady]

Environment info

• NooBaa Version: 5.17
• Platform: NC (running locally on MacOS M1 arch: linux/arm64)

Actual behavior

1. When listing buckets and one of the buckets has an invalid schema the list buckets throw an error of InvalidBucketState.

Expected behavior

1. When listing buckets and one of the buckets has an invalid schema the list of the buckets should appear without the bucket with an invalid schema and print it in the logs.

Steps to reproduce

1. Create an account using the CLI (shira-1001): sudo node src/cmd/manage_nsfs account add --name shira-1001 --new_buckets_path /tmp/nsfs_root1 --access_key <access-key> --secret_key <secret-key> --uid <uid> --gid <gid> Note: before creating the account need to give permission to the new_buckets_path: chmod 777 /tmp/nsfs_root1.
2. Start the NSFS server with: sudo node src/cmd/nsfs --debug 5 Note: before starting the server please add this line: process.env.NOOBAA_LOG_LEVEL = 'nsfs'; in the endpoint.js (before the condition if (process.env.NOOBAA_LOG_LEVEL) {)
3. Create the alias for S3 service: alias s3-nc-user-1='AWS_ACCESS_KEY_ID=<access-key> AWS_SECRET_ACCESS_KEY=<secret-key> aws --no-verify-ssl --endpoint-url https://localhost:6443'.
4. Create 2 buckets in the S3 flow (bucket1, bucket2): s3-nc-user-1-s3 s3 mb s3://bucket1, s3-nc-user-1-s3 s3 mb s3://bucket2
5. List the buckets (should see the 2 buckets): s3-nc-user-1-s3 s3 ls
6. Edit the JSON schema config of one of the buckets (bucket2), for example, add "a":"a" (additional properties are not allowed): sudo vi /etc/noobaa.conf.d/buckets/bucket2.json
7. List the buckets - currently you would receive the InvalidBucketState error: s3-nc-user-1-s3 s3 ls

More information - Screenshots / Logs / Other output

S3 output: An error occurred (InvalidBucketState) when calling the ListBuckets operation: The request is not valid with the current state of the bucket.

NSFS Logs:

Jul-25 11:40:12.941 [nsfs/78985]  [WARN] core.sdk.bucketspace_fs:: list_buckets: read_bucket_sdk_config_info of bucket bucket2 got an error RpcError: must NOT have additional properties | {"additionalProperty":"a"}
    at Object.validate_bucket_schema (/Users/shiradymnik/SourceCode/noobaa-core/src/manage_nsfs/nsfs_schema_utils.js:72:15)
    at BucketSpaceFS.read_bucket_sdk_info (/Users/shiradymnik/SourceCode/noobaa-core/src/sdk/bucketspace_fs.js:135:31)
    at async NsfsObjectSDK._load_bucket_namespace (/Users/shiradymnik/SourceCode/noobaa-core/src/sdk/object_sdk.js:283:41)
    at async LRUCache._load_item (/Users/shiradymnik/SourceCode/noobaa-core/src/util/lru_cache.js:83:22)
    at async LRUCache.get_with_cache (/Users/shiradymnik/SourceCode/noobaa-core/src/util/lru_cache.js:72:13)
    at async NsfsObjectSDK.read_bucket_sdk_config_info (/Users/shiradymnik/SourceCode/noobaa-core/src/sdk/object_sdk.js:180:28)
    at async /Users/shiradymnik/SourceCode/noobaa-core/src/sdk/bucketspace_fs.js:236:26
    at async Semaphore.surround (/Users/shiradymnik/SourceCode/noobaa-core/src/util/semaphore.js:71:84)
    at async Promise.all (index 2)
    at async BucketSpaceFS.list_buckets (/Users/shiradymnik/SourceCode/noobaa-core/src/sdk/bucketspace_fs.js:229:25) {
  rpc_code: 'INVALID_BUCKET_STATE'
}
Jul-25 11:40:12.942 [nsfs/78985] [ERROR] core.endpoint.s3.s3_rest:: S3 ERROR <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidBucketState</Code><Message>The request is not valid with the current state of the bucket.</Message><Resource>/</Resource><RequestId>lz10wk1r-203n9-66m</RequestId></Error> GET / {"host":"localhost:6443","accept-encoding":"identity","user-agent":"aws-cli/2.17.11 md/awscrt#0.20.11 ua/2.0 os/macos#23.4.0 md/arch#arm64 lang/python#3.11.9 md/pyimpl#CPython cfg/retry-mode#standard md/installer#source md/prompt#off md/command#s3.ls","x-amz-date":"20240725T084012Z","x-amz-content-sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","authorization":"AWS4-HMAC-SHA256 Credential=Dwertyuiopasdfg11001/20240725/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=0acfa6c612fffcc8090bf1bdcc388a99dabc4ceda1cfb482cd3dc733de871434"} Error: must NOT have additional properties | {"additionalProperty":"a"}
    at Object.validate_bucket_schema (/Users/shiradymnik/SourceCode/noobaa-core/src/manage_nsfs/nsfs_schema_utils.js:72:15)
    at BucketSpaceFS.read_bucket_sdk_info (/Users/shiradymnik/SourceCode/noobaa-core/src/sdk/bucketspace_fs.js:135:31)
    at async NsfsObjectSDK._load_bucket_namespace (/Users/shiradymnik/SourceCode/noobaa-core/src/sdk/object_sdk.js:283:41)
    at async LRUCache._load_item (/Users/shiradymnik/SourceCode/noobaa-core/src/util/lru_cache.js:83:22)
    at async LRUCache.get_with_cache (/Users/shiradymnik/SourceCode/noobaa-core/src/util/lru_cache.js:72:13)
    at async NsfsObjectSDK.read_bucket_sdk_config_info (/Users/shiradymnik/SourceCode/noobaa-core/src/sdk/object_sdk.js:180:28)
    at async /Users/shiradymnik/SourceCode/noobaa-core/src/sdk/bucketspace_fs.js:236:26
    at async Semaphore.surround (/Users/shiradymnik/SourceCode/noobaa-core/src/util/semaphore.js:71:84)
    at async Promise.all (index 2)
    at async BucketSpaceFS.list_buckets (/Users/shiradymnik/SourceCode/noobaa-core/src/sdk/bucketspace_fs.js:229:25)

[github-actions[bot]]

This issue had no activity for too long - it will now be labeled stale. Update it to prevent it from getting closed.