search

aws / aws-cli

Universal Command Line Interface for Amazon Web Services
Other
15.44k stars 4.1k forks source link

AWSCLI 2.1.29 Python 3.8.8 a bytes-like object is required, not 'str' #6026

Closed genifycom closed 8 months ago

genifycom commented 3 years ago

Confirm by changing [ ] to [x] below to ensure that it's a bug:

Describe the bug aws deploy install --config-file codedeploy.onpremises.yml --override-config

gives Creating the on-premises instance configuration file... DONE Installing the AWS CodeDeploy Agent... ERROR a bytes-like object is required, not 'str' Install the AWS CodeDeploy Agent on the on-premises instance by following the instructions in "Configure Existing On-Premises Instances by Using AWS CodeDeploy" in the AWS CodeDeploy User Guide.

SDK version number AWSCLI 2.1.29 Python 3.8.8

Platform/OS/Hardware/Device Windows Server 2019 Version 1809

To Reproduce (observed behavior)

  1. Downloaded codedeployageent from aws s3 cp s3://aws-codedeploy-us-east-2/latest/codedeploy-agent.msi C:\bvcodedeploy\codedeploy-agent.msi
  2. Ran the msi
  3. Registered the on-premises machine with: aws deploy register --instance-name %computername% --tags Key=%computername%,Value=
  4. Tried to register the configuration with the agent aws deploy install --config-file C:\bvcodedeploy\codedeploy.onpremises.yml --override-config

Expected behavior The bug appears to be a python bug, so I upgraded the aws cli from https://awscli.amazonaws.com/AWSCLIV2-2.0.30.msi from version 2.1.27 to 2.1.29 and tried again with the same result.

Logs/output C:\bvcodedeploy>aws deploy install --config-file C:\bvcodedeploy\codedeploy.onpremises.yml --override-config --debug 2021-03-16 13:49:08,127 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.1.29 Python/3.8.8 Windows/10 exe/AMD64 2021-03-16 13:49:08,127 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['deploy', 'install', '--config-file', 'C:\bvcodedeploy\codedeploy.onpremises.yml', '--override-config', '--debug'] 2021-03-16 13:49:08,158 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_s3 at 0x000001A9F8190A60> 2021-03-16 13:49:08,158 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_ddb at 0x000001A9F8003F70> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method BasicCommand.add_command of <class 'awscli.customizations.configure.configure.ConfigureCommand'>> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function change_name at 0x000001A9F7FA98B0> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function change_name at 0x000001A9F7FB09D0> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function alias_opsworks_cm at 0x000001A9F81A04C0> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_history_commands at 0x000001A9F804EDC0> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method BasicCommand.add_command of <class 'awscli.customizations.devcommands.CLIDevCommand'>> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_waiters at 0x000001A9F8161700> 2021-03-16 13:49:08,174 - MainThread - botocore.loaders - DEBUG - Loading JSON file: C:\Program Files\Amazon\AWSCLIV2\awscli\data\cli.json 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_types at 0x000001A9F80FC940> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function no_sign_request at 0x000001A9F80FD4C0> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_verify_ssl at 0x000001A9F80FD430> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_cli_read_timeout at 0x000001A9F80FD5E0> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_cli_connect_timeout at 0x000001A9F80FD550> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <built-in method update of dict object at 0x000001A9F8233F00> 2021-03-16 13:49:08,174 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.1.29 Python/3.8.8 Windows/10 exe/AMD64 prompt/off 2021-03-16 13:49:08,174 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['deploy', 'install', '--config-file', 'C:\bvcodedeploy\codedeploy.onpremises.yml', '--override-config', '--debug'] 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_timestamp_parser at 0x000001A9F81920D0> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function register_uri_param_handler at 0x000001A9F7575940> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_binary_formatter at 0x000001A9F81FBEE0> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function no_pager_handler at 0x000001A9F7570D30> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x000001A9F75CF4C0> 2021-03-16 13:49:08,174 - MainThread - botocore.utils - DEBUG - IMDS ENDPOINT: http://169.254.169.254/ 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function attach_history_handler at 0x000001A9F804ECA0> 2021-03-16 13:49:08,174 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_json_file_cache at 0x000001A9F8000E50> 2021-03-16 13:49:08,205 - MainThread - botocore.loaders - DEBUG - Loading JSON file: C:\Program Files\Amazon\AWSCLIV2\botocore\data\codedeploy\2014-10-06\service-2.json 2021-03-16 13:49:08,205 - MainThread - botocore.hooks - DEBUG - Event building-command-table.deploy: calling handler <function inject_commands at 0x000001A9F7FA9DC0> 2021-03-16 13:49:08,205 - MainThread - botocore.hooks - DEBUG - Event building-command-table.deploy: calling handler <function add_waiters at 0x000001A9F8161700> 2021-03-16 13:49:08,237 - MainThread - botocore.loaders - DEBUG - Loading JSON file: C:\Program Files\Amazon\AWSCLIV2\botocore\data\codedeploy\2014-10-06\waiters-2.json 2021-03-16 13:49:08,237 - MainThread - botocore.hooks - DEBUG - Event building-command-table.deploy_install: calling handler <function add_waiters at 0x000001A9F8161700> 2021-03-16 13:49:08,237 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.install.config-file: calling handler <awscli.paramfile.URIArgumentHandler object at 0x000001A9F8271790> 2021-03-16 13:49:08,237 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.install: calling handler <awscli.argprocess.ParamShorthandParser object at 0x000001A9F759A7C0> 2021-03-16 13:49:08,237 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.install.override-config: calling handler <awscli.paramfile.URIArgumentHandler object at 0x000001A9F8271790> 2021-03-16 13:49:08,237 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.install: calling handler <awscli.argprocess.ParamShorthandParser object at 0x000001A9F759A7C0> 2021-03-16 13:49:08,237 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.install.agent-installer: calling handler <awscli.paramfile.URIArgumentHandler object at 0x000001A9F8271790> 2021-03-16 13:49:08,237 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env 2021-03-16 13:49:08,237 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role 2021-03-16 13:49:08,237 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role-with-web-identity 2021-03-16 13:49:08,237 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: sso 2021-03-16 13:49:08,237 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file 2021-03-16 13:49:08,237 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials 2021-03-16 13:49:08,252 - MainThread - botocore.loaders - DEBUG - Loading JSON file: C:\Program Files\Amazon\AWSCLIV2\botocore\data\endpoints.json 2021-03-16 13:49:08,252 - MainThread - botocore.hooks - DEBUG - Event choose-service-name: calling handler <function handle_service_name_alias at 0x000001A9F6D9AE50> 2021-03-16 13:49:08,252 - MainThread - botocore.loaders - DEBUG - Loading JSON file: C:\Program Files\Amazon\AWSCLIV2\botocore\data\s3\2006-03-01\service-2.json 2021-03-16 13:49:08,252 - MainThread - botocore.loaders - DEBUG - Loading JSON file: C:\Program Files\Amazon\AWSCLIV2\botocore\data\s3\2006-03-01\service-2.sdk-extras.json 2021-03-16 13:49:08,252 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_post at 0x000001A9F6D4D4C0> 2021-03-16 13:49:08,252 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_url at 0x000001A9F6D4D280> 2021-03-16 13:49:08,252 - MainThread - botocore.endpoint - DEBUG - Setting s3 timeout as (60, 60) Creating the on-premises instance configuration file... DONE 2021-03-16 13:49:24,033 - MainThread - botocore.hooks - DEBUG - Event provide-client-params.s3.GetObject: calling handler <function base64_decode_input_blobs at 0x000001A9F81FE670> 2021-03-16 13:49:24,049 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.GetObject: calling handler <function sse_md5 at 0x000001A9F6DBF040> 2021-03-16 13:49:24,049 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.GetObject: calling handler <function validate_bucket_name at 0x000001A9F6DBBF70> 2021-03-16 13:49:24,049 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.GetObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x000001A9F862BEE0>> 2021-03-16 13:49:24,049 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.GetObject: calling handler <bound method S3ArnParamHandler.handle_arn of <botocore.utils.S3ArnParamHandler object at 0x000001A9F862BFA0>> 2021-03-16 13:49:24,049 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.GetObject: calling handler <function generate_idempotent_uuid at 0x000001A9F6DBBDC0> 2021-03-16 13:49:24,049 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.GetObject: calling handler <function add_expect_header at 0x000001A9F6DBF310> 2021-03-16 13:49:24,049 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.GetObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x000001A9F862BEE0>> 2021-03-16 13:49:24,049 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.GetObject: calling handler <function inject_api_version_header_if_needed at 0x000001A9F6DC0670> 2021-03-16 13:49:24,049 - MainThread - botocore.endpoint - DEBUG - Making request for OperationModel(name=GetObject) with params: {'url_path': '/aws-codedeploy-us-east-1/latest/codedeploy-agent.msi', 'query_string': {}, 'method': 'GET', 'headers': {'User-Agent': 'aws-cli/2.1.29 Python/3.8.8 Windows/10 exe/AMD64 prompt/off command/deploy.install'}, 'body': b'', 'url': 'https://s3.us-east-1.amazonaws.com/aws-codedeploy-us-east-1/latest/codedeploy-agent.msi', 'context': {'client_region': 'us-east-1', 'client_config': <botocore.config.Config object at 0x000001A9F862B340>, 'has_streaming_input': False, 'auth_type': None, 'signing': {'bucket': 'aws-codedeploy-us-east-1'}}} 2021-03-16 13:49:24,049 - MainThread - botocore.hooks - DEBUG - Event request-created.s3.GetObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x000001A9F862B310>> 2021-03-16 13:49:24,049 - MainThread - botocore.hooks - DEBUG - Event choose-signer.s3.GetObject: calling handler <function set_operation_specific_signer at 0x000001A9F6DBBCA0> 2021-03-16 13:49:24,049 - MainThread - botocore.hooks - DEBUG - Event before-sign.s3.GetObject: calling handler <bound method S3EndpointSetter.set_endpoint of <botocore.utils.S3EndpointSetter object at 0x000001A9F866D070>> 2021-03-16 13:49:24,049 - MainThread - botocore.utils - DEBUG - Defaulting to S3 virtual host style addressing with path style addressing fallback. 2021-03-16 13:49:24,049 - MainThread - botocore.utils - DEBUG - Checking for DNS compatible bucket for: https://s3.us-east-1.amazonaws.com/aws-codedeploy-us-east-1/latest/codedeploy-agent.msi 2021-03-16 13:49:24,049 - MainThread - botocore.utils - DEBUG - URI updated to: https://aws-codedeploy-us-east-1.s3.us-east-1.amazonaws.com/latest/codedeploy-agent.msi 2021-03-16 13:49:24,049 - MainThread - botocore.auth - DEBUG - Calculating signature using v4 auth. 2021-03-16 13:49:24,049 - MainThread - botocore.auth - DEBUG - CanonicalRequest: GET /latest/codedeploy-agent.msi

host:aws-codedeploy-us-east-1.s3.us-east-1.amazonaws.com x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 x-amz-date:20210316T174924Z

host;x-amz-content-sha256;x-amz-date e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2021-03-16 13:49:24,049 - MainThread - botocore.auth - DEBUG - StringToSign: AWS4-HMAC-SHA256 20210316T174924Z 20210316/us-east-1/s3/aws4_request 70ba517d0d0feb632ede170f7cb339eea4b0cfbd93d3682d9af95818266f2c4b 2021-03-16 13:49:24,049 - MainThread - botocore.auth - DEBUG - Signature: da06d64963d24b074d8d3cb2c73c4c4b1f640809eac814622dd65bf57dbcde91 2021-03-16 13:49:24,049 - MainThread - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=True, method=GET, url=https://aws-codedeploy-us-east-1.s3.us-east-1.amazonaws.com/latest/codedeploy-agent.msi, headers={'User-Agent': b'aws-cli/2.1.29 Python/3.8.8 Windows/10 exe/AMD64 prompt/off command/deploy.install', 'X-Amz-Date': b'20210316T174924Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAIBXJPTX77DEWKZUA/20210316/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=da06d64963d24b074d8d3cb2c73c4c4b1f640809eac814622dd65bf57dbcde91'}> 2021-03-16 13:49:24,049 - MainThread - botocore.httpsession - DEBUG - Certificate path: C:\Program Files\Amazon\AWSCLIV2\botocore\cacert.pem 2021-03-16 13:49:24,065 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): aws-codedeploy-us-east-1.s3.us-east-1.amazonaws.com:443 2021-03-16 13:49:24,393 - MainThread - urllib3.connectionpool - DEBUG - https://aws-codedeploy-us-east-1.s3.us-east-1.amazonaws.com:443 "GET /latest/codedeploy-agent.msi HTTP/1.1" 200 16696320 2021-03-16 13:49:24,393 - MainThread - botocore.parsers - DEBUG - Response headers: {'x-amz-id-2': 'ya3LmY+yyXnR17ku5S+RqvY+la0tB3g8cIxOqQsdSctVnG09CHDpXlxLfvjfWWrdyEfpZoTygWI=', 'x-amz-request-id': '1M1QM5E11TEC8CVB', 'Date': 'Tue, 16 Mar 2021 17:49:25 GMT', 'Last-Modified': 'Mon, 21 Dec 2020 20:46:55 GMT', 'ETag': '"1cfaf116b6c4fd84ad64d621d6b13a79"', 'x-amz-version-id': '4HB9qshqlFIViUfYuwjSaQOsWoEOXBHB', 'Accept-Ranges': 'bytes', 'Content-Type': '', 'Content-Length': '16696320', 'Server': 'AmazonS3'} 2021-03-16 13:49:24,393 - MainThread - botocore.parsers - DEBUG - Response body: <botocore.response.StreamingBody object at 0x000001A9F86C4190> 2021-03-16 13:49:24,393 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.GetObject: calling handler <bound method RetryHandler.needs_retry of <botocore.retries.standard.RetryHandler object at 0x000001A9F862BE80>> 2021-03-16 13:49:24,393 - MainThread - botocore.retries.standard - DEBUG - Not retrying request. 2021-03-16 13:49:24,393 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.GetObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x000001A9F862BEE0>> 2021-03-16 13:49:24,393 - MainThread - botocore.hooks - DEBUG - Event after-call.s3.GetObject: calling handler <function enhance_error_msg at 0x000001A9F8190CA0> 2021-03-16 13:49:24,393 - MainThread - botocore.hooks - DEBUG - Event after-call.s3.GetObject: calling handler <bound method RetryQuotaChecker.release_retry_quota of <botocore.retries.standard.RetryQuotaChecker object at 0x000001A9F862BA90>> Installing the AWS CodeDeploy Agent... ERROR a bytes-like object is required, not 'str' Install the AWS CodeDeploy Agent on the on-premises instance by following the instructions in "Configure Existing On-Premises Instances by Using AWS CodeDeploy" in the AWS CodeDeploy User Guide.

Additional context Add any other context about the problem here.

kdaily commented 3 years ago

Hi @genifycom,

Thanks for opening this. I see you've encountered this in the past, indicating that it's not a specific issue with the CLI 2.1.29:

https://github.com/aws/aws-codedeploy-agent/issues/215#issuecomment-720130953

We'll look into what the issue could be.

kdaily commented 3 years ago

Hi @genifycom, I'm checking in with the CodeDeploy team about this issue.

kdaily commented 3 years ago

No updates as of yet, @genifycom.

kdaily commented 3 years ago

P45334864

genifycom commented 2 years ago

Currently using aws-cli/2.4.27 Python/3.8.8 Windows/10 exe/AMD64 prompt/off

and same error

Creating the on-premises instance configuration file... DONE Installing the AWS CodeDeploy Agent... ERROR a bytes-like object is required, not 'str'

Thanks

tim-finnigan commented 1 year ago

Hi @genifycom as a workaround we were advised to recommend you:

  1. Register and tag your on-premise instance using aws deploy register
  2. Install Codedeploy Agent following these instructions

Does following those steps resolve the issue for you?

github-actions[bot] commented 8 months ago

Greetings! It looks like this issue hasn’t been active in longer than five days. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.