search

codemagic-ci-cd / cli-tools

Various utilities to managing Android and iOS app builds, code signing, and deployment.
https://codemagic.io/start/
GNU General Public License v3.0
235 stars 39 forks source link

Bugfix: Fail with informational error message in case of broken unicode in CLI args #376

Closed priitlatt closed 7 months ago

priitlatt commented 7 months ago

When invalid unicode sequences are passed to programs via CLI arguments, then down the line number of weird things can happen. Starting from logging issues up to final broken application logic. For example consider the following snippet:

#!/bin/sh

export APP_STORE_APP_ID=1496105355
# note the invalid quotes
app-store-connect apps get “$APP_STORE_APP_ID”

Executing this script will pass application ID as “\udc80\udc9d to Python program, which can cause all sorts of troubles as the value cannot be encoded:

>>> "“\udc80\udc9d".encode()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 1-2: surrogates not allowed
Full erroneous script output ```shell --- Logging error --- Traceback (most recent call last): File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/logging/__init__.py", line 1163, in emit stream.write(msg + self.terminator) UnicodeEncodeError: 'utf-8' codec can't encode characters in position 127-128: surrogates not allowed Call stack: File "/Users/priit/.pyenv/versions/3.12.0/bin/app-store-connect", line 8, in sys.exit(AppStoreConnect.invoke_cli()) File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/cli/cli_app.py", line 201, in invoke_cli cls._log_cli_invoke_started() File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/cli/cli_app.py", line 244, in _log_cli_invoke_started file_logger.debug(Colors.MAGENTA(exec_line)) Message: "\x1b[35mExecute /Users/priit/.pyenv/versions/3.12.0/bin/app-store-connect apps get '“\udc80\udc9d'\x1b[0m" Arguments: () --- Logging error --- Traceback (most recent call last): File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/logging/__init__.py", line 1163, in emit stream.write(msg + self.terminator) UnicodeEncodeError: 'utf-8' codec can't encode characters in position 63-64: surrogates not allowed Call stack: File "/Users/priit/.pyenv/versions/3.12.0/bin/app-store-connect", line 8, in sys.exit(AppStoreConnect.invoke_cli()) File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/cli/cli_app.py", line 213, in invoke_cli CliApp._running_app._invoke_action(args) File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/cli/cli_app.py", line 170, in _invoke_action return cli_action(**action_args) File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/cli/cli_app.py", line 465, in wrapper return func(self, *args, **kwargs) File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/tools/_app_store_connect/action_groups/apps_action_group.py", line 39, in get_app return self._get_resource(application_id, self.api_client.apps, should_print) File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/tools/_app_store_connect/resource_manager_mixin.py", line 57, in _get_resource self.printer.log_get(resource_manager.resource_type, resource_id) File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/tools/_app_store_connect/resource_printer.py", line 101, in log_get self.logger.info(f"Get {resource_type} {resource_id}") Message: 'Get App “\udc80\udc9d' Arguments: () Get App “\udc80\udc9d --- Logging error --- Traceback (most recent call last): File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/logging/__init__.py", line 1163, in emit stream.write(msg + self.terminator) UnicodeEncodeError: 'utf-8' codec can't encode characters in position 103-104: surrogates not allowed Call stack: File "/Users/priit/.pyenv/versions/3.12.0/bin/app-store-connect", line 8, in sys.exit(AppStoreConnect.invoke_cli()) File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/cli/cli_app.py", line 213, in invoke_cli CliApp._running_app._invoke_action(args) File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/cli/cli_app.py", line 170, in _invoke_action return cli_action(**action_args) File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/cli/cli_app.py", line 465, in wrapper return func(self, *args, **kwargs) File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/tools/_app_store_connect/action_groups/apps_action_group.py", line 39, in get_app return self._get_resource(application_id, self.api_client.apps, should_print) File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/tools/_app_store_connect/resource_manager_mixin.py", line 59, in _get_resource resource = read_resource(resource_id) File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/apple/app_store_connect/apps/apps.py", line 67, in read response = self.client.session.get(f"{self.client.API_URL}/apps/{app_id}").json() File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/apple/app_store_connect/api_session.py", line 97, in request return self._do_request(*args, **kwargs) File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/apple/app_store_connect/api_session.py", line 66, in _do_request self._log_request(*request_args, **request_kwargs) File "/Users/priit/.pyenv/versions/3.12.0/lib/python3.12/site-packages/codemagic/apple/app_store_connect/api_session.py", line 42, in _log_request self._logger.info(f">>> {method} {url} {body}") Message: '>>> GET https://api.appstoreconnect.apple.com/v1/apps/“\udc80\udc9d None' Arguments: () GET https://api.appstoreconnect.apple.com/v1/apps/%E2%80%9C%ED%B2%80%ED%B2%9D returned 404: The URL path is not valid - Invalid URL path ```

Avoid this by validating that string arguments can be safely encoded. With the changes introduced in this PR the script fails still, but now with appropriate error message:

usage: app-store-connect [-h] [--log-stream {stderr,stdout}] [--no-color] [--version] [-s] [-v]
                         {app-store-version-localizations,app-store-version-submissions,app-store-versions,apps,beta-app-review-submissions,beta-build-localizations,beta-groups,builds,create-bundle-id,create-certificate,create-profile,delete-bundle-id,delete-certificate,delete-profile,fetch-signing-files,get-bundle-id,get-certificate,get-latest-app-store-build-number,get-latest-build-number,get-latest-testflight-build-number,get-profile,list-builds,list-bundle-id-profiles,list-bundle-ids,list-certificates,list-devices,list-profiles,publish,register-device,review-submission-items,review-submissions}
                         ...
app-store-connect: error: Unknown encoding for argument application_id value: “\udc80\udc9d