pytorch / executorch

On-device AI across mobile, embedded and edge for PyTorch
https://pytorch.org/executorch/
Other
1.94k stars 324 forks source link

The apk downloaded from S3 can not be unzipped #4285

Open guangy10 opened 3 months ago

guangy10 commented 3 months ago

🐛 Describe the bug

Android/test-llama-app (mobile-job) fails due to corrupted apk downloaded from S3. See logs:

Exception: Couldn't upload ci.apk: {'upload': {'arn': 'arn:aws:devicefarm:us-west-2:308535385114:upload:02a2cf0f-6d9b-45ee-ba1a-a086587469e6/9efb530a-d0ab-4776-96b8-b3ef3cd25848', 'name': 'mobile-job-android-9954110435-1-2024-07-16-ZOfaUHWN_ci.apk', 'created': datetime.datetime(2024, 7, 16, 9, 47, 10, 174000, tzinfo=tzlocal()), 'type': 'ANDROID_APP', 'status': 'FAILED', 'url': 'https://prod-us-west-2-uploads.s3-us-west-2.amazonaws.com/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A308535385114%3Aproject%3A02a2cf0f-6d9b-45ee-ba1a-a086587469e6/uploads/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A308535385114%3Aupload%3A02a2cf0f-6d9b-45ee-ba1a-a086587469e6/9efb530a-d0ab-4776-96b8-b3ef3cd25848/mobile-job-android-9954110435-1-2024-07-16-ZOfaUHWN_ci.apk?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20240716T094715Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=AKIAUJHLTYS5AWNTRO6L%2F20240716%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=bbb9bdbd372237f3b03f75537d392dcd4cd1f1bc1dcce11fcdf135d4c8e62f74', 'metadata': '{"errorMessageUrl":"https://docs.aws.amazon.com/console/devicefarm/ANDROID_APP_UNZIP_FAILED","errorMessage":"We could not unzip your APK file. Please verify that the file is valid and try again.","errorCode":"ANDROID_APP_UNZIP_FAILED"}', 'contentType': 'application/octet-stream', 'category': 'PRIVATE'}, 'ResponseMetadata': {'RequestId': '1cf266cf-ca19-4389-a9ea-f19d73d2d628', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '1cf266cf-ca19-4389-a9ea-f19d73d2d628', 'content-type': 'application/x-amz-json-1.1', 'content-length': '1235', 'date': 'Tue, 16 Jul 2024 09:47:14 GMT'}, 'RetryAttempts': 0}}

You can follow the S3 link and download it manually to reproduce the issue by running unzip mobile-job-android-9954110435-1-2024-07-16-ZOfaUHWN_ci.apk, you will see detailed error like

Archive:  mobile-job-android-9954110435-1-2024-07-16-ZOfaUHWN_ci.apk
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of mobile-job-android-9954110435-1-2024-07-16-ZOfaUHWN_ci.apk or
        mobile-job-android-9954110435-1-2024-07-16-ZOfaUHWN_ci.apk.zip, and cannot find mobile-job-android-9954110435-1-2024-07-16-ZOfaUHWN_ci.apk.ZIP, period.

I tired build the same apk locally and it's just unzipped fine. So I think the issue is either the uploading to S3 or downloading from S3 is not done correctly.

Versions

torch 2.5.0.dev20240618+cpu

huydhn commented 3 months ago

The issue with the invalid S3 path can be fixed with something like https://github.com/pytorch/executorch/pull/4289