Improve mechanism of finding cache artifacts

[canhth]

Problems

Not sure if I missed something else, but I have a problem with finding the cache artifacts in my real project.

• I've set cache_commit_history to 100 but the case about missing the cache artifacts frequently happens when we have a lot of new commits.
• If increase the cache_commit_history value to a big number, I think it might slower for pod install step.

The job run in CICD machines

• My master branch will run producer mode twice a day, but the team size is >50 members, and the branches are updated frequently.
• Team A created a new Feature A branch from Commit 1.
• In the ~50 first new commit from branch Feature A, we're able to find the artifacts.
• In the next > 100 commits, we will get this message for pod install no artifacts available

Expected/desired behavior

• Feature A branch should always can use the cache generated from Commit 1 in master branch.
• We set primary_branch, should we only find the commits in primary_branch branch only?

  Running this command: git log -50 --pretty=format:%H --first-parent > gitlog.txt, the --first-parent option only show the merged commits new branch created. I think we can improve this one?

Relevant integration setup

[x ] CocoaPods cocoapods-xcremotecache plugin

[polac24]

Hi! Finding the most recent commit sha with artifacts is indeed a lengthy process - it makes a HEAD request for all commits one-by-one. If you generate artifacts so seldom (even I suggest running that more often than twice a day), a history count should be higher.

Regarding the image, it works a bit differently. It starts counting since a first common sha, not from a current sha (note we call something like git log -50 --pretty=format:%H --first-parent COMMON_SHA_HERE. Here is the code that finds the COMMON_SHA: https://github.com/spotify/XCRemoteCache/blob/b6318e97855f8f23b2dc670a8765baad80340d25/Sources/XCRemoteCache/Git/GitClient.swift#L76-L85

Do you see room for implement?

[canhth]

Thank you for clarifying.

If you generate artifacts so seldom (even I suggest running that more often than twice a day), a history count should be higher.

• I think this will cost my team a lot of CI resources, my project has 3 different build configurations, and also support both iphoneos & simulator. This means we need to run 6 jobs for one producer mode. Twice a day is 12 jobs. Each job needs ~10-15mins to complete. My team is quite big, and we have a lot of new branches like "Feature A".

Starting the most recent commit with the primary branch (master) check 100 commits if has artifacts ready Do you see room for implement?

I'm sorry but I'm a bit confused here, could you please help to point out which one is correct:

1. You mean it starts counting since a first common sha and then the result will be only the commits history of master branch? (merge-base commit only)

- Commit 4            - Commit A2
- Commit 3            - Commit A1
- Commit 2     --> - Checkout from commit 2 
- Commit 1   
- Commit 0
- Commit -1
** master **          ** feature A **

After run git log -50 --pretty=format:%H --first-parent COMMON_SHA, the list will be:

- Commit 2
- Commit 1   
- Commit 0
- Commit -1
-........

Is that correct? And no need to implement anything else because searching commit history from master branch already supported? I ran this in my project and the most of the commits in master branch.

2. "Starting the most recent commit with the primary branch (master) check 100 commits if has artifacts ready" is not ready yet? And we can support this option?

[polac24]

XCRemoteCache works according to the scenario 1. so no need to implement anything.