spotify / XCRemoteCache

Other
827 stars 50 forks source link

URL to get artifacts contains the wrong configuration for some targets #109

Open samuelsainz opened 2 years ago

samuelsainz commented 2 years ago

My integration setup

Expected/desired behavior

I'm building the app with a build configuration named "Debug Enterprise"—correctly selected in the scheme that I'm building. Sometimes xcprebuild get the artifacts without problems building the URL like this:

https://artifactory.svc.appname.com/artifactory/mobile-ios-xcremotecache/v1/beta/meta/d1fdee5c8b5540b44341a61b1b57ed7d328dd1bf-TargetNameX-Debug%20Enterprise-iphonesimulator-13C100-4b7e99cf543c0cbccbd4df6a462ca769.json

But in some cases I am seeing the next error: Network request failed with unsuccessful code 404. Looking at the logs I saw that for those artifacts xcprebuild is using the wrong URL to get the artifact—containing a wrong configuration:

https://artifactory.svc.appname.com/artifactory/mobile-ios-xcremotecache/v1/beta/meta/d1fdee5c8b5540b44341a61b1b57ed7d328dd1bf-TargetNameY-Debug-iphonesimulator-13C100-192149891c1951efa6af7f819e0d90bc.json

The configuration is Debug instead of Debug%20Enterprise.

Minimal reproduction of the problem with instructions

It happens randomly, I wasn't able to figure out how to reproduce it. Sometimes it happens for all the Artifacts and some times it happens just for some of them.

Producer Logs

N/A

Consumer Logs

2022-03-25 13:06:43.285 E  xcprebuild[80070:32b42c] (ZIPFoundation) Couldn't verify if should disable RC for d1fdee5c8b5540b44341a61b1b57ed7d328dd1bf.
2022-03-25 13:06:43.302 I  xcprebuild[80070:32b42c] (ZIPFoundation) Found url to remapp: file:///Users/samuelsainz/Library/Developer/Xcode/DerivedData/App-dfrnrjyugdqbaydkvxawxpsalegp/Build/Products/. Remapping: /Users/samuelsainz/Library/Developer/Xcode/DerivedData/App-dfrnrjyugdqbaydkvxawxpsalegp/Build/Products
2022-03-25 13:06:43.302 I  xcprebuild[80070:32b42c] (ZIPFoundation) Found url to remapp: file:///Users/samuelsainz/dev/appname/appname-ios/App/Pods/. Remapping: /Users/samuelsainz/dev/appname/appname-ios/App/Pods
2022-03-25 13:06:43.302 I  xcprebuild[80070:32b42c] (ZIPFoundation) Found url to remapp: file:///Users/samuelsainz/dev/appname/appname-ios/App/Modules/. Remapping: /Users/samuelsainz/dev/appname/appname-ios/App/Modules
2022-03-25 13:06:43.302 I  xcprebuild[80070:32b42c] (ZIPFoundation) Making request https://artifactory.svc.appname.com/artifactory/mobile-ios-xcremotecache/v1/beta/meta/d1fdee5c8b5540b44341a61b1b57ed7d328dd1bf-ZIPFoundation-Debug%20Enterprise-iphonesimulator-13C100-cd4d4a21dc0386892af9e88337d556c1.json
2022-03-25 13:06:50.496 I  xcprebuild[80070:32b42c] (ZIPFoundation) Downloading artifact to file:///Users/samuelsainz/Library/Developer/Xcode/DerivedData/App-dfrnrjyugdqbaydkvxawxpsalegp/Build/Intermediates.noindex/ZIPFoundation.build/Debug%20Enterprise-iphonesimulator/ZIPFoundation.build/xccache/3ff1096f0416ed0c11339a0d48500e3c.zip
2022-03-25 13:06:52.447 I  xcprebuild[80070:32b42c] (ZIPFoundation) Artifact unzipped to file:///Users/samuelsainz/Library/Developer/Xcode/DerivedData/App-dfrnrjyugdqbaydkvxawxpsalegp/Build/Intermediates.noindex/ZIPFoundation.build/Debug%20Enterprise-iphonesimulator/ZIPFoundation.build/xccache/3ff1096f0416ed0c11339a0d48500e3c
2022-03-25 13:06:59.008 I  xcswiftc[80189:32b7db] Optional .swiftinterface file not found in the artifact at: /Users/samuelsainz/Library/Developer/Xcode/DerivedData/App-dfrnrjyugdqbaydkvxawxpsalegp/Build/Intermediates.noindex/ZIPFoundation.build/Debug Enterprise-iphonesimulator/ZIPFoundation.build/Objects-normal/x86_64/ZIPFoundation.swiftinterface
2022-03-25 13:06:59.435 I  xcpostbuild[80263:32b8d9] (ZIPFoundation) Found url to remapp: file:///Users/samuelsainz/Library/Developer/Xcode/DerivedData/App-dfrnrjyugdqbaydkvxawxpsalegp/Build/Products/. Remapping: /Users/samuelsainz/Library/Developer/Xcode/DerivedData/App-dfrnrjyugdqbaydkvxawxpsalegp/Build/Products
2022-03-25 13:06:59.435 I  xcpostbuild[80263:32b8d9] (ZIPFoundation) Found url to remapp: file:///Users/samuelsainz/dev/appname/appname-ios/App/Pods/. Remapping: /Users/samuelsainz/dev/appname/appname-ios/App/Pods
2022-03-25 13:06:59.435 I  xcpostbuild[80263:32b8d9] (ZIPFoundation) Found url to remapp: file:///Users/samuelsainz/dev/appname/appname-ios/App/Modules/. Remapping: /Users/samuelsainz/dev/appname/appname-ios/App/Modules
2022-03-25 13:10:17.907 E  xcprebuild[87675:332a00] (ZIPFoundation) Couldn't verify if should disable RC for d1fdee5c8b5540b44341a61b1b57ed7d328dd1bf.
2022-03-25 13:10:17.921 I  xcprebuild[87675:332a00] (ZIPFoundation) Found url to remapp: file:///Users/samuelsainz/Library/Developer/Xcode/DerivedData/App-dfrnrjyugdqbaydkvxawxpsalegp/Index/Build/Products/. Remapping: /Users/samuelsainz/Library/Developer/Xcode/DerivedData/App-dfrnrjyugdqbaydkvxawxpsalegp/Index/Build/Products
2022-03-25 13:10:17.921 I  xcprebuild[87675:332a00] (ZIPFoundation) Found url to remapp: file:///Users/samuelsainz/dev/appname/appname-ios/App/Pods/. Remapping: /Users/samuelsainz/dev/appname/appname-ios/App/Pods
2022-03-25 13:10:17.921 I  xcprebuild[87675:332a00] (ZIPFoundation) Found url to remapp: file:///Users/samuelsainz/dev/appname/appname-ios/App/Modules/. Remapping: /Users/samuelsainz/dev/appname/appname-ios/App/Modules
2022-03-25 13:10:17.922 I  xcprebuild[87675:332a00] (ZIPFoundation) Making request https://artifactory.svc.appname.com/artifactory/mobile-ios-xcremotecache/v1/beta/meta/d1fdee5c8b5540b44341a61b1b57ed7d328dd1bf-ZIPFoundation-Debug-iphonesimulator-13C100-68d2a4af57d31af4c9b95158fa9c700c.json
2022-03-25 13:10:18.940 I  xcprebuild[87675:332aae] (ZIPFoundation) Network request failed with unsuccessful code 404
2022-03-25 13:10:18.941 E  xcprebuild[87675:332a00] (ZIPFoundation) Prebuild step failed with error: unsuccessfulResponse(status: 404)
2022-03-25 13:10:20.212 I  xcpostbuild[87763:332cc2] (ZIPFoundation) Found url to remapp: file:///Users/samuelsainz/Library/Developer/Xcode/DerivedData/App-dfrnrjyugdqbaydkvxawxpsalegp/Index/Build/Products/. Remapping: /Users/samuelsainz/Library/Developer/Xcode/DerivedData/App-dfrnrjyugdqbaydkvxawxpsalegp/Index/Build/Products
2022-03-25 13:10:20.212 I  xcpostbuild[87763:332cc2] (ZIPFoundation) Found url to remapp: file:///Users/samuelsainz/dev/appname/appname-ios/App/Pods/. Remapping: /Users/samuelsainz/dev/appname/appname-ios/App/Pods
2022-03-25 13:10:20.212 I  xcpostbuild[87763:332cc2] (ZIPFoundation) Found url to remapp: file:///Users/samuelsainz/dev/appname/appname-ios/App/Modules/. Remapping: /Users/samuelsainz/dev/appname/appname-ios/App/Modules
 

Environment

Post build stats

hit_count: 152
miss_count: 0
local_cache_bytes: 515153920
indexing_hit_count: 0
indexing_miss_count: 156

Others

I tried to debug this but I am struggling with debugger not stoping at the breakpoints, I already tried everything in #103

When I expand the [XCRC] Prebuild TargetName script for one of these targets (using the hamburger button in Xcode log) the export command for CONFIGURATION var is "Debug\ Enterprise" as expected.

I tried also using the exclude_build_configurations config for the plugin (adding 'Debug' to that list) but it didn't work.

I appreciate any help you could give me to troubleshoot this. Thanks in advance

polac24 commented 2 years ago

Hello! The second error comes from the Indexing build - probably Debug is the configuration that Xcode uses. Xcode 13 introduced a new feature: indexing arena that builds the target in isolation and its output is used to power only Xcode's indexing.

You can preview commands used in the indexing arena in Xcode (report navigator tab), but first you need to enable it (Xcode restart needed):

defaults write com.apple.dt.Xcode IDEIndexShowLog -bool YES
polac24 commented 2 years ago

To troubleshoot exclude_build_configurations, can you check your Xcode's project for the ZIPFoundation and double check that the SWIFT_EXEC build setting is not set for the Debug configuration (most likely empty). I would expect something like that:

Screenshot 2022-03-26 at 09 54 41
samuelsainz commented 2 years ago

It seems that the Debug configuration is correctly excluded, I am seeing the swiftc value for it which makes sense:

Screen Shot 2022-04-11 at 12 12 51

But it is still trying to get "Debug" artifacts as well looking at the logs.

This is not critical though since for the right configuration I am getting 100% cache hit, but I am wondering why is doing that and if it affects the performance somehow.

polac24 commented 2 years ago

Get it: to have a clean hit rate (we have separated indexing hit counts anyway) and not unnecessary generate HTTP requests, we could add some king of bail out logic to the XcodeProjIntegrate when a configuration that should be disabled has been recognized (e.g. comparing CONFIGURATION env in the shell script).

Contribution PR is welcome!