spotify / XCRemoteCache

Other
827 stars 50 forks source link

XCRemoteCache's cache management in producer side #104

Closed dwirandyh closed 2 years ago

dwirandyh commented 2 years ago

Hallo I have setup my producer in CI and use the cache from consumer, but i wonder how xcremotecache manage old artifact that generated before? Will xcremotecache automatically delete artifacts that are no longer in use?

Relevant integration setup

[x] CocoaPods cocoapods-xcremotecache plugin

polac24 commented 2 years ago

Hello! No, XCRemoteCache does not have any logic to evict old artifacts on a remote side, it should be managed by the storage server.

XCRemoteCache works with any HTTP(S) servers and some of them have such a feature available. For instance GCS offers Object Lifecycle Management.

dwirandyh commented 2 years ago

hmm, i see, i just wondering, how about artifact_maximum_age configuration? does it evict old artifact?

polac24 commented 2 years ago

artifact_maximum_age controls a local cache only (the one in ~/Library/Caches/XCRemoteCache/)

artifact_maximum_age Maximum age in days HTTP response should be locally cached before being evicted
cezarsignori commented 2 years ago

Hi! This is not in the scope of the original question but matches the title. Please let me know if a separate ticket is best.

Why does the producer mode not download unchanged build artifacts like the consumer?

It looks like the producer could do faster builds by building and uploading only the changes.

polac24 commented 2 years ago

Good question: reusing previous artifacts on a producer side exists but it was never battle tested. To enable it, you can set producer-fast mode. Sidenote: even the list here doesn't document it, we have it only documented for xcprepare integrate here.

What is risky?

I think that XCRemoteCache could observe extra build settings (for settings without absolute that custom_fingerprint_envs should be enough) to fully support that. There is a change that some random configuration could be missed and thus XCRemoteCache could just overcache a target.

If you can run producer-fast trial, I am really curious about results.

polac24 commented 2 years ago

The original problem seems to be resolved.