Get caches failed after changed project directory

[canhth]

My integration setup

• CocoaPods cocoapods-xcremotecache plugin
• Contain 97 dependencies, including Development Pods.

Problem & Expectation

• Local machine: cache hit rate is 80/97. But after moved to another directory, the cache hit rate is 40/97.

• CI machine, each build job has a different directory and the cache hit rate is 40/100.

Expectation:

1. Change the source directory path should not affect the cache hit rate.
2. CI jobs with different directory will hit the same cache as local machine.

Minimal reproduction of the problem with instructions To reproduce this locally, we can test by this steps:

1. Provide producer then run consumer mode.
2. Count the total number of cached.
3. Move your project to another directory.
4. Delete Pod folder and rerun consumer mode.
5. Count the total number of consumer mode and compare.

Environment

• cocoapods-xcremotecache: 0.0.9
• Xcode: 13.2.1

Notes

I've read Build artifacts portability in Spotify blog and understand XCRemoteCache already support fingerprint_override_extension & debug-prefix-map. Not sure why this not working for our case, it blocking us to apply XCRemoteCache to CICD system.

[polac24]

Hello! That looks like your meta contains some absolute paths. Please follow the following steps to review them: https://github.com/spotify/XCRemoteCache/issues/69#issuecomment-1029274283 If that is a case, you could take advantage of custom_rewrite_envs or out_of_band_mappings. These .rcinfo properties allow integrating XCRemoteCache if .xcodeproj customized build settings.

[canhth]

Hi @polac24,

I've added 'custom_rewrite_envs' => ['PODS_TARGET_SRCROOT'] to Pod file, and the hit rate was increased a bit. But there are still some meta files contains absolute paths:

Maybe my Development Pods are too complex? Local Pod A is a dependency of Local Pod B?

• Tested with disable/enabled generate_multiple_pod_projects.
• Examples:

  $(BUILD_DIR)\/Dev Release-iphoneos\/lottie-ios\/Lottie.framework\/Modules\/Lottie.swiftmodule\/arm64-apple-ios.swiftmodule.md5",
  "\/Users\/dev\/ci\/ios-project\/2a0cc62bfe2ea1b43b16f72d8b7b9936b2c7d92c\/1652497703\/Modules\/FaceDetection\/FaceDetection\/Frameworks\/MGBaseKit.framework\/Headers\/LicenseManager.h",
  "\/Users\/dev\/ci\/ios-project\/2a0cc62bfe2ea1b43b16f72d8b7b9936b2c7d92c\/1652497703\/Modules\/FaceDetection\/FaceDetection\/Frameworks\/MGBaseKit.framework\/Headers\/MGAnimation.h",
  "\/Users\/dev\/ci\/ios-project\/2a0cc62bfe2ea1b43b16f72d8b7b9936b2c7d92c\/1652497703\/Modules\/FaceDetection\/FaceDetection\/Frameworks\/MGLivenessDetection.framework\/Headers\/MGBaseBottomManager.h",
  "\/Users\/dev\/ci\/ios-project\/2a0cc62bfe2ea1b43b16f72d8b7b9936b2c7d92c\/1652497703\/Modules\/FaceDetection\/FaceDetection\/Frameworks\/MGLivenessDetection.framework\/Headers\/MGBaseCountdownView.h",

"dependencies":
["\/Users\/dev\/Library\/Developer\/Xcode\/DerivedData\/MyProject-ftuxmakcdjlvgzaqsizziqkphcrp\/Build\/Intermediates.noindex\/ArchiveIntermediates\/MyProject\/IntermediateBuildFilesPath\/UninstalledProducts\/iphoneos\/Alamofire.framework\/Headers\/Alamofire-Swift.h",
"$(BUILD_DIR)\/Dev Release-iphoneos\/Alamofire\/Alamofire.framework\/Modules\/Alamofire.swiftmodule\/arm64-apple-ios.swiftmodule.md5",
"\/Users\/dev\/Library\/Developer\/Xcode\/DerivedData\/MyProject-ftuxmakcdjlvgzaqsizziqkphcrp\/Build\/Intermediates.noindex\/ArchiveIntermediates\/MyProject\/IntermediateBuildFilesPath\/UninstalledProducts\/iphoneos\/AlignedCollectionViewFlowLayout.framework\/Headers\/AlignedCollectionViewFlowLayout-Swift.h",
"$(BUILD_DIR)\/Dev Release-iphoneos\/AlignedCollectionViewFlowLayout\/AlignedCollectionViewFlowLayout.framework\/Modules\/AlignedCollectionViewFlowLayout.swiftmodule\/arm64-apple-

[polac24]

Hello! Regarding the first example, can you provide more context? From which target's meta is that snippet? MGLivenessDetection? What is 2a0cc62bfe2ea1b43b16f72d8b7b9936b2c7d92c, 1652497703. From the second example, seems you archive the target (not build). That is definitely a bug but without access to a project it is hard to guess which ENV (aka build settings in Xcode) exactly is not replacing the absolute path to the generic placeholder.

The logic is as follows:

• On a producer side, there is a command line xcpostbuild step that replaces several ENVs into placeholders (link)
• You could inspect all build settings (which are then converted into the process ENVs) in Xcode:

  or from a similar xcodebuild output.

For instance, for the second example, you can review the given target's ENVs and see which of them contain: \/Users\/dev\/Library\/Developer\/Xcode\/DerivedData\/MyProject-ftuxmakcdjlvgzaqsizziqkphcrp\/Build\/Intermediates.noindex\/ArchiveIntermediates\/MyProject\/IntermediateBuildFilesPath\/UninstalledProducts.

Maybe my Development Pods are too complex?

We haven't tested that but don't see any blockers for that.

Finally, having a sample project would help me investigating.

[polac24]

Have another tip: #67 mentions that maybe we should include BUILT_PRODUCTS_DIR in custom_rewrite_envs. Maybe that shortcoming surfaces in your scenario?

[canhth]

Hi @polac24 , Thank you so much for your quick response during the weekend. I really appreciate that.

• 2a0cc62bfe2ea1b43b16f72d8b7b9936b2c7d92c is the commit and 1652497703 is timestamp, to separate the directory folder in every CI build job.
• Yes, this is archive (not build).
• It's quite hard to create a sample project to reproduce the issue, but let me try to create one if your suggestion won't work.

Based on the description in https://github.com/spotify/XCRemoteCache/pull/67

But on archive, they are different, and DependencyProcessorImpl uses TARGET_BUILD_DIR to determine ownership of artifacts.

Should I include TARGET_BUILD_DIR to the custom_rewrite_envs? or just BUILT_PRODUCTS_DIR can work in both build and archive?

[canhth]

Added 'custom_rewrite_envs' => ['PODS_TARGET_SRCROOT', 'BUILT_PRODUCTS_DIR', 'TARGET_BUILD_DIR'], and the meta file not contains absolute path anymore. (replaced with $(TARGET_BUILD_DIR))

BUT the problem still remains, our CI could not hit the cache for the Archive job for some other core framework Maybe this case is not supported yet. Because I still see absolute path for this case: Pod A and Pod B are in the same root directory path. Pod B is nested of Pod A:

- /Modules/PodA/ --> PodA.spec
- /Modules/PodA/Core/ --> PodB.spec

• Local Pod A:

  ss.exclude_files = 'PodAPath/Classes/Core/**/*'
  ss.ios.dependency 'PodB', '~> 0.1.0'

• Local Pod B:

  s.source_files = 'PodAPath/Classes/Core/**/*'

  And the meta file of PodB contains absolute path of it's source code directory. (like the example above.)

I checked the build log and there are several ENVs containing the path related to pod: PODS_BUILD_DIR, PODS_CONFIGURATION_BUILD_DIR, PODS_ROOT, PODS_TARGET_SRCROOT. Can we included one of those to custom_rewrite_envs?

There is a Limitation for cocoapod-plugin:

When generate_multiple_pod_projects mode is enabled, only first-party targets are cached by XCRemoteCache (all dependencies are compiled locally). I've disabled generate_multiple_pod_projects flag already.

[polac24]

I cannot reproduce it, here is the sample project I generated, and even for an archive, I didn't get absolute paths: https://github.com/polac24/nestedCocoapodsSample

When generate_multiple_pod_projects mode is enabled, only first-party targets are cached by XCRemoteCache (all dependencies are compiled locally).

That has been resolved in #57. We need to update the doc, thanks for noticing that.

[canhth]

@polac24 Thanks for your update. I can't reproduce the issue in that example as well. Maybe my project has some configuration that causes the mismatch. But for that issue, I can fix it by not using nested pods.

My cache hit rate still does not exceed 100% because there is still one LocalPod that generates absolute path if using vendored_frameworks with framework NOT .xcframework. Ex: s.ios.vendored_frameworks = 'FaceDetection/Frameworks/*.framework'

Here is the example I forked from your repo: https://github.com/canhth/nestedCocoapodsSample

[canhth]

Thanks @polac24, Tried with your repo: https://github.com/spotify/XCRemoteCache/pull/136 and it works perfectly 👍