codemagic-ci-cd / cli-tools

Various utilities to managing Android and iOS app builds, code signing, and deployment.
https://codemagic.io/start/
GNU General Public License v3.0
243 stars 42 forks source link

Add support for large iOS application packages x2 #356

Closed priitlatt closed 11 months ago

priitlatt commented 11 months ago

Follow up to PR #342.

Extracting files from ipa files can fail with Bad magic number for file header error even if the binary is exceptionally large (>4GB).

Full stacktrace:

[18:38:23 29-09-2023] WARNING cli_app.py:114 > Executing AppStoreConnect action publish failed unexpectedly. Detailed logs are available at "/var/folders/vs/tcrc5cns67zgynxt6fssjdg80000gn/T/codemagic-29-09-23.log". To see more details about the error, add `--verbose` command line option.
[18:38:23 29-09-2023] ERROR cli_app.py:116 > Exception traceback:
Traceback (most recent call last):
  File "/Users/priit/nevercode/cli-tools/src/codemagic/models/application_package/ipa.py", line 25, in _validate_package
    return bool(self.info_plist)
                ^^^^^^^^^^^^^^^
  File "/Users/priit/nevercode/cli-tools/src/codemagic/models/application_package/ipa.py", line 97, in info_plist
    return self._get_info_plist()
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/priit/nevercode/cli-tools/src/codemagic/models/application_package/ipa.py", line 91, in _get_info_plist
    info_plist_contents = self._get_app_file_contents("Info.plist")
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/priit/nevercode/cli-tools/src/codemagic/models/application_package/ipa.py", line 71, in _get_app_file_contents
    return self._extract_file(filename_filter)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/priit/nevercode/cli-tools/src/codemagic/models/application_package/ipa.py", line 59, in _extract_file
    completed_process = subprocess.run(extract_command, capture_output=True, check=True)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/priit/.pyenv/versions/3.11.2/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('unzip', '-p', PosixPath('zip_error.ipa'), 'Payload/Runner.app/Info.plist')' returned non-zero exit status 2.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/priit/nevercode/cli-tools/src/codemagic/cli/cli_app.py", line 206, in invoke_cli
    CliApp._running_app._invoke_action(args)
  File "/Users/priit/nevercode/cli-tools/src/codemagic/cli/cli_app.py", line 163, in _invoke_action
    return cli_action(**action_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/priit/nevercode/cli-tools/src/codemagic/cli/cli_app.py", line 458, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/priit/nevercode/cli-tools/src/codemagic/tools/_app_store_connect/actions/publish_action.py", line 305, in publish
    application_packages = self._get_publishing_application_packages(application_package_path_patterns)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/priit/nevercode/cli-tools/src/codemagic/tools/_app_store_connect/actions/publish_action.py", line 505, in _get_publishing_application_packages
    application_package: Union[Ipa, MacOsPackage] = Ipa(path)
                                                    ^^^^^^^^^
  File "/Users/priit/nevercode/cli-tools/src/codemagic/models/application_package/abstract_package.py", line 19, in __init__
    self._validate_package()
  File "/Users/priit/nevercode/cli-tools/src/codemagic/models/application_package/ipa.py", line 27, in _validate_package
    raise IOError(f"Not a valid iOS application package at {self.path}") from error
OSError: Not a valid iOS application package at zip_error.ipa

It is known that unzip is not able to handle files this big, but 7z is capable of extracting components from there. Use 7z whenever it is available in place of unzip.