search

ceph / s3-tests

Compatibility tests for S3 clones
MIT License
297 stars 286 forks source link

S3 Function Header tests seems faulty (executed on ceph vm cluster) #438

Open sumedhak27 opened 2 years ago

sumedhak27 commented 2 years ago

Faulty Tests:

Issue Description: The above-mentioned tests are failing on ceph cluster deployed in the virtual machine environment. When the same tests are performed using REST API, we get the appropriate response. (403 Forbidden)

Ceph/S3-tests Failure Output The test is to set the Authorization header to "" (empty), from the debug request header log we can see that the Authorization header was not set to "". ```bash $ S3TEST_CONF=./s3tests.motr.conf virtualenv/bin/nosetests s3tests_boto3.functional.test_headers:test_bucket_create_bad_authorization_empty -v 2>&1 | tee -a cephtest.log s3tests_boto3.functional.test_headers.test_bucket_create_bad_authorization_empty ... FAIL ====================================================================== FAIL: s3tests_boto3.functional.test_headers.test_bucket_create_bad_authorization_empty ---------------------------------------------------------------------- Traceback (most recent call last): File "/root/ceph-tests/s3-tests/virtualenv/lib/python3.6/site-packages/nose/case.py", line 198, in runTest self.test(*self.arg) File "/root/ceph-tests/s3-tests/s3tests_boto3/functional/test_headers.py", line 500, in test_bucket_create_bad_authorization_empty e = _add_header_create_bad_bucket(headers) File "/root/ceph-tests/s3-tests/s3tests_boto3/functional/test_headers.py", line 113, in _add_header_create_bad_bucket e = assert_raises(ClientError, client.create_bucket, Bucket=bucket_name) File "/root/ceph-tests/s3-tests/s3tests_boto3/functional/utils.py", line 21, in assert_raises raise AssertionError("%s not raised" % excName) AssertionError: ClientError not raised -------------------- >> begin captured logging << -------------------- botocore.hooks: DEBUG: Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane botocore.hooks: DEBUG: Changing event name from before-call.apigateway to before-call.api-gateway botocore.hooks: DEBUG: Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict botocore.hooks: DEBUG: Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration botocore.hooks: DEBUG: Changing event name from before-parameter-build.route53 to before-parameter-build.route-53 botocore.hooks: DEBUG: Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search botocore.hooks: DEBUG: Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section botocore.hooks: DEBUG: Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask botocore.hooks: DEBUG: Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section botocore.hooks: DEBUG: Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search botocore.hooks: DEBUG: Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section botocore.loaders: DEBUG: Loading JSON file: /root/ceph-tests/s3-tests/virtualenv/lib/python3.6/site-packages/botocore/data/endpoints.json botocore.loaders: DEBUG: Loading JSON file: /root/ceph-tests/s3-tests/virtualenv/lib/python3.6/site-packages/botocore/data/sdk-default-configuration.json botocore.hooks: DEBUG: Event choose-service-name: calling handler botocore.loaders: DEBUG: Loading JSON file: /root/ceph-tests/s3-tests/virtualenv/lib/python3.6/site-packages/botocore/data/s3/2006-03-01/service-2.json botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler ._handler at 0x7f007fe9e730> botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler botocore.endpoint: DEBUG: Setting s3 timeout as (60, 60) botocore.loaders: DEBUG: Loading JSON file: /root/ceph-tests/s3-tests/virtualenv/lib/python3.6/site-packages/botocore/data/_retry.json botocore.client: DEBUG: Registering retry handlers for service: s3 botocore.hooks: DEBUG: Event choose-service-name: calling handler botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler ._handler at 0x7f007fe9e730> botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler botocore.endpoint: DEBUG: Setting s3 timeout as (60, 60) botocore.client: DEBUG: Registering retry handlers for service: s3 botocore.hooks: DEBUG: Event choose-service-name: calling handler botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler ._handler at 0x7f007fe9e730> botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler botocore.endpoint: DEBUG: Setting s3 timeout as (60, 60) botocore.client: DEBUG: Registering retry handlers for service: s3 botocore.hooks: DEBUG: Event before-parameter-build.s3.ListBuckets: calling handler botocore.hooks: DEBUG: Event before-parameter-build.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event before-parameter-build.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event before-parameter-build.s3.ListBuckets: calling handler botocore.hooks: DEBUG: Event before-call.s3.ListBuckets: calling handler botocore.hooks: DEBUG: Event before-call.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event before-call.s3.ListBuckets: calling handler botocore.hooks: DEBUG: Event before-call.s3.ListBuckets: calling handler botocore.endpoint: DEBUG: Making request for OperationModel(name=ListBuckets) with params: {'url_path': '/', 'query_string': '', 'method': 'GET', 'headers': {'User-Agent': 'Boto3/1.21.33 Python/3.6.8 Linux/4.18.0-305.3.1.el8_4.x86_64 Botocore/1.24.33'}, 'body': b'', 'url': 'http://s3.seagate.com:8000/', 'context': {'client_region': 'us-east-1', 'client_config': , 'has_streaming_input': False, 'auth_type': None, 'signing': {'bucket': None}}} botocore.hooks: DEBUG: Event request-created.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event choose-signer.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event choose-signer.s3.ListBuckets: calling handler botocore.hooks: DEBUG: Event before-sign.s3.ListBuckets: calling handler > botocore.utils: DEBUG: Using S3 path style addressing. botocore.auth: DEBUG: Calculating signature using v4 auth. botocore.auth: DEBUG: CanonicalRequest: GET / host:s3.seagate.com:8000 x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 x-amz-date:20220405T112900Z host;x-amz-content-sha256;x-amz-date e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 botocore.auth: DEBUG: StringToSign: AWS4-HMAC-SHA256 20220405T112900Z 20220405/us-east-1/s3/aws4_request 22228fc1e8b134ddb0eb200e0ad20cba21136163ec54d59d8cce3de3cf9f3ece botocore.auth: DEBUG: Signature: 3c4a33971d3a168f1df5024128387f34a21901337e6c0b9c400e725f6b538a06 botocore.hooks: DEBUG: Event request-created.s3.ListBuckets: calling handler botocore.endpoint: DEBUG: Sending http request: urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): s3.seagate.com:8000 urllib3.connectionpool: DEBUG: http://s3.seagate.com:8000 "GET / HTTP/1.1" 200 None botocore.parsers: DEBUG: Response headers: {'Transfer-Encoding': 'chunked', 'Content-Type': 'application/xml', 'Date': 'Tue, 05 Apr 2022 11:29:00 GMT', 'Connection': 'Keep-Alive'} botocore.parsers: DEBUG: Response body: b'mgwuserSuperUseremptybkt2022-03-07T08:43:27.486Zhehe2022-04-04T12:34:53.828Zmybkt2022-02-24T05:35:23.667Zmybkt22022-03-04T05:46:30.031Ztestbkt2022-02-24T11:47:30.855Ztestbkt12022-04-04T05:56:54.843Ztestbkt22022-02-24T12:16:52.286Z' botocore.hooks: DEBUG: Event needs-retry.s3.ListBuckets: calling handler botocore.retryhandler: DEBUG: No retry needed. botocore.hooks: DEBUG: Event needs-retry.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event before-parameter-build.s3.ListBuckets: calling handler botocore.hooks: DEBUG: Event before-parameter-build.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event before-parameter-build.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event before-parameter-build.s3.ListBuckets: calling handler botocore.hooks: DEBUG: Event before-call.s3.ListBuckets: calling handler botocore.hooks: DEBUG: Event before-call.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event before-call.s3.ListBuckets: calling handler botocore.hooks: DEBUG: Event before-call.s3.ListBuckets: calling handler botocore.endpoint: DEBUG: Making request for OperationModel(name=ListBuckets) with params: {'url_path': '/', 'query_string': '', 'method': 'GET', 'headers': {'User-Agent': 'Boto3/1.21.33 Python/3.6.8 Linux/4.18.0-305.3.1.el8_4.x86_64 Botocore/1.24.33'}, 'body': b'', 'url': 'http://s3.seagate.com:8000/', 'context': {'client_region': 'us-east-1', 'client_config': , 'has_streaming_input': False, 'auth_type': None, 'signing': {'bucket': None}}} botocore.hooks: DEBUG: Event request-created.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event choose-signer.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event choose-signer.s3.ListBuckets: calling handler botocore.hooks: DEBUG: Event before-sign.s3.ListBuckets: calling handler > botocore.utils: DEBUG: Using S3 path style addressing. botocore.auth: DEBUG: Calculating signature using v4 auth. botocore.auth: DEBUG: CanonicalRequest: GET / host:s3.seagate.com:8000 x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 x-amz-date:20220405T112900Z host;x-amz-content-sha256;x-amz-date e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 botocore.auth: DEBUG: StringToSign: AWS4-HMAC-SHA256 20220405T112900Z 20220405/us-east-1/s3/aws4_request 22228fc1e8b134ddb0eb200e0ad20cba21136163ec54d59d8cce3de3cf9f3ece botocore.auth: DEBUG: Signature: 3c4a33971d3a168f1df5024128387f34a21901337e6c0b9c400e725f6b538a06 botocore.hooks: DEBUG: Event request-created.s3.ListBuckets: calling handler botocore.endpoint: DEBUG: Sending http request: urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): s3.seagate.com:8000 urllib3.connectionpool: DEBUG: http://s3.seagate.com:8000 "GET / HTTP/1.1" 200 None botocore.parsers: DEBUG: Response headers: {'Transfer-Encoding': 'chunked', 'Content-Type': 'application/xml', 'Date': 'Tue, 05 Apr 2022 11:29:00 GMT', 'Connection': 'Keep-Alive'} botocore.parsers: DEBUG: Response body: b'mgwuserSuperUseremptybkt2022-03-07T08:43:27.486Zhehe2022-04-04T12:34:53.828Zmybkt2022-02-24T05:35:23.667Zmybkt22022-03-04T05:46:30.031Ztestbkt2022-02-24T11:47:30.855Ztestbkt12022-04-04T05:56:54.843Ztestbkt22022-02-24T12:16:52.286Z' botocore.hooks: DEBUG: Event needs-retry.s3.ListBuckets: calling handler botocore.retryhandler: DEBUG: No retry needed. botocore.hooks: DEBUG: Event needs-retry.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event before-parameter-build.s3.ListBuckets: calling handler botocore.hooks: DEBUG: Event before-parameter-build.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event before-parameter-build.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event before-parameter-build.s3.ListBuckets: calling handler botocore.hooks: DEBUG: Event before-call.s3.ListBuckets: calling handler botocore.hooks: DEBUG: Event before-call.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event before-call.s3.ListBuckets: calling handler botocore.hooks: DEBUG: Event before-call.s3.ListBuckets: calling handler botocore.endpoint: DEBUG: Making request for OperationModel(name=ListBuckets) with params: {'url_path': '/', 'query_string': '', 'method': 'GET', 'headers': {'User-Agent': 'Boto3/1.21.33 Python/3.6.8 Linux/4.18.0-305.3.1.el8_4.x86_64 Botocore/1.24.33'}, 'body': b'', 'url': 'http://s3.seagate.com:8000/', 'context': {'client_region': 'us-east-1', 'client_config': , 'has_streaming_input': False, 'auth_type': None, 'signing': {'bucket': None}}} botocore.hooks: DEBUG: Event request-created.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event choose-signer.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event choose-signer.s3.ListBuckets: calling handler botocore.hooks: DEBUG: Event before-sign.s3.ListBuckets: calling handler > botocore.utils: DEBUG: Using S3 path style addressing. botocore.auth: DEBUG: Calculating signature using v4 auth. botocore.auth: DEBUG: CanonicalRequest: GET / host:s3.seagate.com:8000 x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 x-amz-date:20220405T112900Z host;x-amz-content-sha256;x-amz-date e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 botocore.auth: DEBUG: StringToSign: AWS4-HMAC-SHA256 20220405T112900Z 20220405/us-east-1/s3/aws4_request 22228fc1e8b134ddb0eb200e0ad20cba21136163ec54d59d8cce3de3cf9f3ece botocore.auth: DEBUG: Signature: 3c4a33971d3a168f1df5024128387f34a21901337e6c0b9c400e725f6b538a06 botocore.hooks: DEBUG: Event request-created.s3.ListBuckets: calling handler botocore.endpoint: DEBUG: Sending http request: urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): s3.seagate.com:8000 urllib3.connectionpool: DEBUG: http://s3.seagate.com:8000 "GET / HTTP/1.1" 200 None botocore.parsers: DEBUG: Response headers: {'Transfer-Encoding': 'chunked', 'Content-Type': 'application/xml', 'Date': 'Tue, 05 Apr 2022 11:29:00 GMT', 'Connection': 'Keep-Alive'} botocore.parsers: DEBUG: Response body: b'mgwuserSuperUseremptybkt2022-03-07T08:43:27.486Zhehe2022-04-04T12:34:53.828Zmybkt2022-02-24T05:35:23.667Zmybkt22022-03-04T05:46:30.031Ztestbkt2022-02-24T11:47:30.855Ztestbkt12022-04-04T05:56:54.843Ztestbkt22022-02-24T12:16:52.286Z' botocore.hooks: DEBUG: Event needs-retry.s3.ListBuckets: calling handler botocore.retryhandler: DEBUG: No retry needed. botocore.hooks: DEBUG: Event needs-retry.s3.ListBuckets: calling handler > botocore.hooks: DEBUG: Event choose-service-name: calling handler botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler ._handler at 0x7f007fe9e730> botocore.hooks: DEBUG: Event creating-client-class.s3: calling handler botocore.endpoint: DEBUG: Setting s3 timeout as (60, 60) botocore.client: DEBUG: Registering retry handlers for service: s3 botocore.hooks: DEBUG: Event before-parameter-build.s3.CreateBucket: calling handler botocore.hooks: DEBUG: Event before-parameter-build.s3.CreateBucket: calling handler > botocore.hooks: DEBUG: Event before-parameter-build.s3.CreateBucket: calling handler > botocore.hooks: DEBUG: Event before-parameter-build.s3.CreateBucket: calling handler botocore.hooks: DEBUG: Event before-call.s3.CreateBucket: calling handler . at 0x7f007f50f620> botocore.hooks: DEBUG: Event before-call.s3.CreateBucket: calling handler botocore.hooks: DEBUG: Event before-call.s3.CreateBucket: calling handler > botocore.hooks: DEBUG: Event before-call.s3.CreateBucket: calling handler botocore.hooks: DEBUG: Event before-call.s3.CreateBucket: calling handler botocore.endpoint: DEBUG: Making request for OperationModel(name=CreateBucket) with params: {'url_path': '/sumedh-t1ssyn1xpcjl2mzh4mcq9e-1', 'query_string': {}, 'method': 'PUT', 'headers': {'User-Agent': 'Boto3/1.21.33 Python/3.6.8 Linux/4.18.0-305.3.1.el8_4.x86_64 Botocore/1.24.33', 'Authorization': ''}, 'body': b'', 'url': 'http://s3.seagate.com:8000/sumedh-t1ssyn1xpcjl2mzh4mcq9e-1', 'context': {'client_region': 'us-east-1', 'client_config': , 'has_streaming_input': False, 'auth_type': None, 'signing': {'bucket': 'sumedh-t1ssyn1xpcjl2mzh4mcq9e-1'}}} botocore.hooks: DEBUG: Event request-created.s3.CreateBucket: calling handler > botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler > botocore.hooks: DEBUG: Event choose-signer.s3.CreateBucket: calling handler botocore.hooks: DEBUG: Event before-sign.s3.CreateBucket: calling handler > botocore.utils: DEBUG: Using S3 path style addressing. botocore.auth: DEBUG: Calculating signature using v4 auth. botocore.auth: DEBUG: CanonicalRequest: PUT /sumedh-t1ssyn1xpcjl2mzh4mcq9e-1 host:s3.seagate.com:8000 x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 x-amz-date:20220405T112900Z host;x-amz-content-sha256;x-amz-date e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 botocore.auth: DEBUG: StringToSign: AWS4-HMAC-SHA256 20220405T112900Z 20220405/us-east-1/s3/aws4_request 1d0b623b039917f15061a92204eb94923edb3679bd5efa933460d4f5c9b1766b botocore.auth: DEBUG: Signature: 9eea05bf0bb3688f5af7fecc1b328533b50451bc7bf20e835ab39f8b0154cd5d botocore.hooks: DEBUG: Event request-created.s3.CreateBucket: calling handler botocore.endpoint: DEBUG: Sending http request: urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): s3.seagate.com:8000 urllib3.connectionpool: DEBUG: http://s3.seagate.com:8000 "PUT /sumedh-t1ssyn1xpcjl2mzh4mcq9e-1 HTTP/1.1" 200 0 botocore.parsers: DEBUG: Response headers: {'Content-Length': '0', 'Date': 'Tue, 05 Apr 2022 11:29:00 GMT', 'Connection': 'Keep-Alive'} botocore.parsers: DEBUG: Response body: b'' botocore.hooks: DEBUG: Event needs-retry.s3.CreateBucket: calling handler botocore.retryhandler: DEBUG: No retry needed. botocore.hooks: DEBUG: Event needs-retry.s3.CreateBucket: calling handler > --------------------- >> end captured logging << --------------------- ---------------------------------------------------------------------- Ran 1 test in 0.796s FAILED (failures=1) (virtualenv) ```
Test output using REST API ```bash ~/ceph-client ⌚ 4:29:18 $ cd /root/ceph-client ; /usr/bin/env /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-2.el8_5.x86_64/bin/java -ag entlib:jdwp=transport=dt_socket,server=n,suspend=y,address=localhost:34569 -cp /tmp/cp_bbidnu5w7per3qn9hi9pyfa63 .jar com.seagate.ceph_client.CephAdminAPIClient Date: Mon, 4 Apr 2022 10:35:34 GMT HeaderString: PUT Mon, 4 Apr 2022 10:35:34 GMT /hehe Auth: HTTP/1.1 403 ForbiddenResponse Content is: {"Code":"AccessDenied","BucketName":"hehe","RequestId":"tx000008d3648d298f720bc-00624ac9f7-1050-default","HostId":"1050-default-default"} ```
romayalon commented 8 months ago

Hi @sumedhak27 Did you find out what went wrong causing these tests to fail? I started seeing these tests failing today