search

aws / aws-cli

Universal Command Line Interface for Amazon Web Services
Other
15.56k stars 4.13k forks source link

(AccessDenied) when calling the ListBuckets operation: Access Denied #8506

Closed fmappin closed 9 months ago

fmappin commented 9 months ago

Describe the bug

I am trying to use the awscli to access s3 buckets that I have permissions to access, however when i do so it gives me (access denied). According to the debug it recognizes my credentials, and according to my permissions I should have access to these buckets.

running both :aws sts get-caller-identity aws iam get-user This indicate It is aware who I am . specfic error : An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied

Expected Behavior

i expected it to list the buckets I have access to, that i can view via the web console.

Current Behavior

cessDenied) when calling the ListBuckets operation: Access Denied fmappin@MacBook-Air ~ % aws s3 ls --debug 2024-01-25 12:59:36,539 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.15.14 Python/3.11.6 Darwin/23.2.0 exe/x86_64 2024-01-25 12:59:36,539 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['s3', 'ls', '--debug'] 2024-01-25 12:59:36,552 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_s3 at 0x10e0e2a20> 2024-01-25 12:59:36,553 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_ddb at 0x10df0e980> 2024-01-25 12:59:36,553 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method BasicCommand.add_command of <class 'awscli.customizations.configure.configure.ConfigureCommand'>> 2024-01-25 12:59:36,553 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function change_name at 0x10de6e5c0> 2024-01-25 12:59:36,553 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function change_name at 0x10de96020> 2024-01-25 12:59:36,553 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function alias_opsworks_cm at 0x10e0f14e0> 2024-01-25 12:59:36,553 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_history_commands at 0x10df6d3a0> 2024-01-25 12:59:36,553 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method BasicCommand.add_command of <class 'awscli.customizations.devcommands.CLIDevCommand'>> 2024-01-25 12:59:36,553 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_waiters at 0x10e0f13a0> 2024-01-25 12:59:36,553 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method AliasSubCommandInjector.on_building_command_table of <awscli.alias.AliasSubCommandInjector object at 0x10e1af950>> 2024-01-25 12:59:36,553 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/data/cli.json 2024-01-25 12:59:36,554 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_types at 0x10e016d40> 2024-01-25 12:59:36,554 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function no_sign_request at 0x10e017060> 2024-01-25 12:59:36,554 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_verify_ssl at 0x10e016fc0> 2024-01-25 12:59:36,554 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_cli_read_timeout at 0x10e0171a0> 2024-01-25 12:59:36,554 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_cli_connect_timeout at 0x10e017100> 2024-01-25 12:59:36,554 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <built-in method update of dict object at 0x10e1acd80> 2024-01-25 12:59:36,555 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.15.14 Python/3.11.6 Darwin/23.2.0 exe/x86_64 prompt/off 2024-01-25 12:59:36,555 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['s3', 'ls', '--debug'] 2024-01-25 12:59:36,555 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_timestamp_parser at 0x10e0e3380> 2024-01-25 12:59:36,555 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function register_uri_param_handler at 0x10d78aac0> 2024-01-25 12:59:36,555 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_binary_formatter at 0x10e176c00> 2024-01-25 12:59:36,555 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function no_pager_handler at 0x10d67cd60> 2024-01-25 12:59:36,555 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x10d7a23e0> 2024-01-25 12:59:36,560 - MainThread - botocore.utils - DEBUG - IMDS ENDPOINT: http://169.254.169.254/ 2024-01-25 12:59:36,566 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function attach_history_handler at 0x10df51c60> 2024-01-25 12:59:36,566 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_json_file_cache at 0x10def6c00> 2024-01-25 12:59:36,566 - MainThread - botocore.hooks - DEBUG - Event building-command-table.s3: calling handler <function add_waiters at 0x10e0f13a0> 2024-01-25 12:59:36,566 - MainThread - botocore.hooks - DEBUG - Event building-command-table.s3: calling handler <bound method AliasSubCommandInjector.on_building_command_table of <awscli.alias.AliasSubCommandInjector object at 0x10e1af950>> 2024-01-25 12:59:36,567 - MainThread - botocore.hooks - DEBUG - Event building-command-table.s3_ls: calling handler <function add_waiters at 0x10e0f13a0> 2024-01-25 12:59:36,567 - MainThread - botocore.hooks - DEBUG - Event building-command-table.s3_ls: calling handler <bound method AliasSubCommandInjector.on_building_command_table of <awscli.alias.AliasSubCommandInjector object at 0x10e1af950>> 2024-01-25 12:59:36,567 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.paths: calling handler <awscli.paramfile.URIArgumentHandler object at 0x10ce29110> 2024-01-25 12:59:36,567 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.anonymous: calling handler <awscli.paramfile.URIArgumentHandler object at 0x10ce29110> 2024-01-25 12:59:36,567 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.page-size: calling handler <awscli.paramfile.URIArgumentHandler object at 0x10ce29110> 2024-01-25 12:59:36,567 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.human-readable: calling handler <awscli.paramfile.URIArgumentHandler object at 0x10ce29110> 2024-01-25 12:59:36,567 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.ls: calling handler <awscli.argprocess.ParamShorthandParser object at 0x10d6a1d50> 2024-01-25 12:59:36,567 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.summarize: calling handler <awscli.paramfile.URIArgumentHandler object at 0x10ce29110> 2024-01-25 12:59:36,567 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.ls: calling handler <awscli.argprocess.ParamShorthandParser object at 0x10d6a1d50> 2024-01-25 12:59:36,567 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.request-payer: calling handler <awscli.paramfile.URIArgumentHandler object at 0x10ce29110> 2024-01-25 12:59:36,568 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env 2024-01-25 12:59:36,568 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role 2024-01-25 12:59:36,568 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role-with-web-identity 2024-01-25 12:59:36,568 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: sso 2024-01-25 12:59:36,568 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file 2024-01-25 12:59:36,568 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials 2024-01-25 12:59:36,569 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/botocore/data/endpoints.json 2024-01-25 12:59:36,581 - MainThread - botocore.hooks - DEBUG - Event choose-service-name: calling handler <function handle_service_name_alias at 0x10c31a660> 2024-01-25 12:59:36,596 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/botocore/data/s3/2006-03-01/service-2.json 2024-01-25 12:59:36,613 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/botocore/data/s3/2006-03-01/endpoint-rule-set-1.json 2024-01-25 12:59:36,618 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/botocore/data/partitions.json 2024-01-25 12:59:36,620 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_post at 0x10c268900> 2024-01-25 12:59:36,620 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_url at 0x10c268680> 2024-01-25 12:59:36,620 - MainThread - botocore.configprovider - DEBUG - Looking for endpoint for s3 via: environment_service 2024-01-25 12:59:36,620 - MainThread - botocore.configprovider - DEBUG - Looking for endpoint for s3 via: environment_global 2024-01-25 12:59:36,620 - MainThread - botocore.configprovider - DEBUG - Looking for endpoint for s3 via: config_service 2024-01-25 12:59:36,620 - MainThread - botocore.configprovider - DEBUG - Looking for endpoint for s3 via: config_global 2024-01-25 12:59:36,620 - MainThread - botocore.configprovider - DEBUG - No configured endpoint found. 2024-01-25 12:59:36,653 - MainThread - botocore.endpoint - DEBUG - Setting s3 timeout as (60, 60) 2024-01-25 12:59:36,655 - MainThread - botocore.utils - DEBUG - Registering S3 region redirector handler 2024-01-25 12:59:36,655 - MainThread - botocore.utils - DEBUG - Registering S3Express Identity Resolver 2024-01-25 12:59:36,655 - MainThread - botocore.hooks - DEBUG - Event before-endpoint-resolution.s3: calling handler <function customize_endpoint_resolver_builtins at 0x10c342de0> 2024-01-25 12:59:36,655 - MainThread - botocore.hooks - DEBUG - Event before-endpoint-resolution.s3: calling handler <bound method S3RegionRedirectorv2.redirect_from_cache of <botocore.utils.S3RegionRedirectorv2 object at 0x112a4e890>> 2024-01-25 12:59:36,655 - MainThread - botocore.regions - DEBUG - Calling endpoint provider with parameters: {'Region': 'us-east-1', 'UseFIPS': False, 'UseDualStack': False, 'ForcePathStyle': False, 'Accelerate': False, 'UseGlobalEndpoint': False, 'DisableMultiRegionAccessPoints': False, 'UseArnRegion': True} 2024-01-25 12:59:36,656 - MainThread - botocore.regions - DEBUG - Endpoint provider result: https://s3.us-east-1.amazonaws.com 2024-01-25 12:59:36,656 - MainThread - botocore.regions - DEBUG - Selecting from endpoint provider's list of auth schemes: "sigv4". User selected auth scheme is: "None" 2024-01-25 12:59:36,656 - MainThread - botocore.regions - DEBUG - Selected auth type "v4" as "v4" with signing context params: {'region': 'us-east-1', 'signing_name': 's3', 'disableDoubleEncoding': True} 2024-01-25 12:59:36,656 - MainThread - botocore.hooks - DEBUG - Event provide-client-params.s3.ListBuckets: calling handler <function base64_decode_input_blobs at 0x10e176ca0> 2024-01-25 12:59:36,656 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.ListBuckets: calling handler <function validate_bucket_name at 0x10c340b80> 2024-01-25 12:59:36,656 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.ListBuckets: calling handler <function remove_bucket_from_url_paths_from_model at 0x10c342c00> 2024-01-25 12:59:36,656 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.ListBuckets: calling handler <bound method S3RegionRedirectorv2.annotate_request_context of <botocore.utils.S3RegionRedirectorv2 object at 0x112a4e890>> 2024-01-25 12:59:36,656 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.ListBuckets: calling handler <bound method S3ExpressIdentityResolver.inject_signing_cache_key of <botocore.utils.S3ExpressIdentityResolver object at 0x112a4f2d0>> 2024-01-25 12:59:36,656 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.ListBuckets: calling handler <function generate_idempotent_uuid at 0x10c3409a0> 2024-01-25 12:59:36,656 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.ListBuckets: calling handler <function add_expect_header at 0x10c340f40> 2024-01-25 12:59:36,656 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.ListBuckets: calling handler <bound method S3ExpressIdentityResolver.apply_signing_cache_key of <botocore.utils.S3ExpressIdentityResolver object at 0x112a4f2d0>> 2024-01-25 12:59:36,656 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.ListBuckets: calling handler <function inject_api_version_header_if_needed at 0x10c342480> 2024-01-25 12:59:36,656 - MainThread - botocore.endpoint - DEBUG - Making request for OperationModel(name=ListBuckets) with params: {'url_path': '/', 'query_string': '', 'method': 'GET', 'headers': {'User-Agent': 'aws-cli/2.15.14 Python/3.11.6 Darwin/23.2.0 exe/x86_64 prompt/off command/s3.ls'}, 'body': b'', 'url': 'https://s3.us-east-1.amazonaws.com/', 'context': {'client_region': 'us-east-1', 'client_config': <botocore.config.Config object at 0x10e1b1910>, 'has_streaming_input': False, 'auth_type': 'v4', 'signing': {'region': 'us-east-1', 'signing_name': 's3', 'disableDoubleEncoding': True}, 'endpoint_properties': {'authSchemes': [{'disableDoubleEncoding': True, 'name': 'sigv4', 'signingName': 's3', 'signingRegion': 'us-east-1'}]}, 's3_redirect': {'redirected': False, 'bucket': None, 'params': {}}}} 2024-01-25 12:59:36,657 - MainThread - botocore.hooks - DEBUG - Event request-created.s3.ListBuckets: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x10d714710>> 2024-01-25 12:59:36,657 - MainThread - botocore.hooks - DEBUG - Event choose-signer.s3.ListBuckets: calling handler <function set_operation_specific_signer at 0x10c340860> 2024-01-25 12:59:36,657 - MainThread - botocore.hooks - DEBUG - Event before-sign.s3.ListBuckets: calling handler <function remove_arn_from_signing_path at 0x10c342d40> 2024-01-25 12:59:36,657 - MainThread - botocore.hooks - DEBUG - Event before-sign.s3.ListBuckets: calling handler <bound method S3ExpressIdentityResolver.resolve_s3express_identity of <botocore.utils.S3ExpressIdentityResolver object at 0x112a4f2d0>> 2024-01-25 12:59:36,657 - MainThread - botocore.auth - DEBUG - Calculating signature using v4 auth. 2024-01-25 12:59:36,657 - MainThread - botocore.auth - DEBUG - CanonicalRequest: GET /

host:s3.us-east-1.amazonaws.com x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 x-amz-date:20240125T175936Z

host;x-amz-content-sha256;x-amz-date e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2024-01-25 12:59:36,657 - MainThread - botocore.auth - DEBUG - StringToSign: AWS4-HMAC-SHA256 20240125T175936Z 20240125/us-east-1/s3/aws4_request 216c40eff7097d73a320eb4ca14dc92ee6b315ee462429df29715c41d0964bdc 2024-01-25 12:59:36,658 - MainThread - botocore.auth - DEBUG - Signature: a0623578a6f6897f2eef06332151f0da7b38f1cccd62d7ccbc43f098076d2d98 2024-01-25 12:59:36,658 - MainThread - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=GET, url=https://s3.us-east-1.amazonaws.com/, headers={'User-Agent': b'aws-cli/2.15.14 Python/3.11.6 Darwin/23.2.0 exe/x86_64 prompt/off command/s3.ls', 'X-Amz-Date': b'20240125T175936Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAVWFNYRA7OBQ6W4WQ/20240125/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=a0623578a6f6897f2eef06332151f0da7b38f1cccd62d7ccbc43f098076d2d98'}> 2024-01-25 12:59:36,658 - MainThread - botocore.httpsession - DEBUG - Certificate path: /usr/local/aws-cli/awscli/botocore/cacert.pem 2024-01-25 12:59:36,659 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): s3.us-east-1.amazonaws.com:443 2024-01-25 12:59:37,022 - MainThread - urllib3.connectionpool - DEBUG - https://s3.us-east-1.amazonaws.com:443 "GET / HTTP/1.1" 403 None 2024-01-25 12:59:37,023 - MainThread - botocore.parsers - DEBUG - Response headers: {'x-amz-request-id': '5APR09ZBD6ENPW7R', 'x-amz-id-2': '9s5zuOqejWk6TG4S4S5flxLRIihqDWZiUC0aA/wW6w+lPJ+Fe2yUZMo6o4zUpMLSZfc+AX47pg4=', 'Content-Type': 'application/xml', 'Transfer-Encoding': 'chunked', 'Date': 'Thu, 25 Jan 2024 17:59:36 GMT', 'Server': 'AmazonS3'} 2024-01-25 12:59:37,024 - MainThread - botocore.parsers - DEBUG - Response body: b'<?xml version="1.0" encoding="UTF-8"?>\nAccessDeniedAccess Denied5APR09ZBD6ENPW7R9s5zuOqejWk6TG4S4S5flxLRIihqDWZiUC0aA/wW6w+lPJ+Fe2yUZMo6o4zUpMLSZfc+AX47pg4=' 2024-01-25 12:59:37,027 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.ListBuckets: calling handler <bound method RetryHandler.needs_retry of <botocore.retries.standard.RetryHandler object at 0x112a4eed0>> 2024-01-25 12:59:37,027 - MainThread - botocore.retries.standard - DEBUG - Not retrying request. 2024-01-25 12:59:37,027 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.ListBuckets: calling handler <bound method S3RegionRedirectorv2.redirect_from_error of <botocore.utils.S3RegionRedirectorv2 object at 0x112a4e890>> 2024-01-25 12:59:37,027 - MainThread - botocore.hooks - DEBUG - Event after-call.s3.ListBuckets: calling handler <function enhance_error_msg at 0x10e0e2fc0> 2024-01-25 12:59:37,028 - MainThread - botocore.hooks - DEBUG - Event after-call.s3.ListBuckets: calling handler <bound method RetryQuotaChecker.release_retry_quota of <botocore.retries.standard.RetryQuotaChecker object at 0x112a4e9d0>> 2024-01-25 12:59:37,028 - MainThread - awscli.clidriver - DEBUG - Exception caught in main() Traceback (most recent call last): File "awscli/clidriver.py", line 460, in main File "awscli/customizations/commands.py", line 151, in call File "awscli/customizations/commands.py", line 205, in call File "awscli/customizations/s3/subcommands.py", line 506, in _run_main File "awscli/customizations/s3/subcommands.py", line 571, in _list_all_buckets File "awscli/botocore/client.py", line 357, in _api_call File "awscli/botocore/client.py", line 724, in _make_api_call botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied

An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied

Reproduction Steps

aws s3 ls

aws s3 ls --debug

Possible Solution

I tried uninstalling and reinstalling the awscli , the issue continued. I have no solutions, only problems.

Additional Information/Context

Python 3.12.1 (v3.12.1:2305ca5144, Dec 7 2023, 17:23:38) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin Type "help", "copyright", "credits" or "license" for more information.

import boto3 s3 = boto3.client('s3') s3.list_buckets() Traceback (most recent call last): File "", line 1, in File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/botocore/client.py", line 553, in _api_call return self._make_api_call(operation_name, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/botocore/client.py", line 1009, in _make_api_call raise error_class(parsed_response, operation_name) botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied

CLI version used

aws-cli/2.15.14 Python/3.11.6 Darwin/23.2.0 exe/x86_64 prompt/off

Environment details (OS name and version, etc.)

macOS Sonoma Version 14.2.1

github-actions[bot] commented 9 months ago

This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.