spotify / XCRemoteCache

Other
825 stars 50 forks source link

Want to understand what's the best value for artifact_maximum_age for CI runs. #235

Open anuragstar opened 5 months ago

anuragstar commented 5 months ago

My integration setup

[ ] CocoaPods cocoapods-xcremotecache plugin [x] Automatic integration using xcprepare integrate ... [ ] Manual integration [ ] Carthage

Expected/desired behavior @polac24 We are trying to run XCremote cache for our project want to understand what's the best value to configure for artifact_maximum_age on CI

Minimal reproduction of the problem with instructions

Producer Logs

 [REPLACE THIS WITH YOUR INFORMATION] 

Consumer Logs

 [REPLACE THIS WITH YOUR INFORMATION] 

Pods/Carthage file

 [REPLACE THIS WITH YOUR INFORMATION] 

Environment

Post build stats

 [REPLACE THIS WITH YOUR INFORMATION] 

Others

polac24 commented 4 months ago

artifact_maximum_age is introduced to cleanup you local cache at ~/Library/Caches/XCRemoteCache/. To find the best value, you should find the sweet spot between local disk cache storage and limiting network traffic load.

In practice, the default 30d should be a long enough time and I wouldn't suggest to worry about it, unless the size of ~/Library/Caches/XCRemoteCache/ skyrockets.

So if you set artifact_maximum_age to 30, as a very early step of xcprepare, XCRemoteCache iterates over all artifacts available locally (~/Library/Caches/XCRemoteCache/) and removes files that have been deleted more than 30d ago. As the location suggests, that is only a local cache: if an artifact file is removed locally but still required, it will be redownloaded on-demand.

anuragstar commented 4 months ago

@polac24 want to understand if it is cache on the CI machine - ~/Library/Caches/XCRemoteCache/, If the XCRemote fetches few artifacts, will the new artifacts will be overrided ?, there won't be any risk builds will be compiled from local caches present on the CI machines ?

polac24 commented 4 months ago

~/Library/Caches/XCRemoteCache/ is just a mirror of all GET requests made in the consumer mode. If you use the consumer mode on CI, it will be populated on CI too. Note that only consumer mode downloads artifacts. Producer, as name suggests, always build artifacts from scratch.

If the XCRemote fetches few artifacts, will the new artifacts will be overrided ? it will not download an artifacts if is available locally

there won't be any risk builds will be compiled from local caches present on the CI machines ? XCRemoteCache is not aware of CI/local, only consumer/producer. Probably you mean CI=producer - then no risk, as producer never reads anything from local or remote cache.