noobaa / noobaa-core

High-performance S3 application gateway to any backend - file / s3-compatible / multi-clouds / caching / replication ...
https://www.noobaa.io
Apache License 2.0
273 stars 80 forks source link

s3api error improvements when FS is unmounted or degraded #8541

Open anandhu-karattu opened 2 days ago

anandhu-karattu commented 2 days ago

Environment info

Actual behavior

If the FS is unmounted on the protocol node and CES IP is still active on that node, s3api will try to access the buckets using the same CES IP. So we need to provide proper error messages to the user if the FS is down or degraded.

For eg: In below cases buckets exists , however the FS is unmounted on the node where CES IP is currently active.

  1. List buckets [root@fa-akarattu-mgmt-host ~]# account7000 s3api list-buckets urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.0.100.119'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings {     "Buckets": [],     "Owner": {         "DisplayName": "NooBaa",         "ID": "123"     } }

  2. List contents of the bucket [root@fa-akarattu-mgmt-host ~]# account7000 s3 ls s3://anandhu1000 urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.0.100.119'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings

An error occurred (NoSuchBucket) when calling the ListObjectsV2 operation: The specified bucket does not exist.

Expected behavior

We should check the status of FS and throw an error accordingly.

Steps to reproduce

  1. Create a bucket
  2. Make the FS down or degraded in the node where CES IP is residing (no HA, just the FS is inaccessible)
  3. Access the bucket via s3api or s3 using the CES IP.

More information - Screenshots / Logs / Other output