Providing IREE Snapshot "releases" as a branch

[GMNGeoffrey]

Chatting with @hcindyl about integrating IREE into a project that uses git-repo. They're interested in having a less frequently updated version of IREE than ToT. Not necessarily stable, but updated less frequently. We discussed using the snapshot release, which creates a release twice daily. The wrinkle is that git-repo apparently has spotty support for fetching from tags, so they'd like a branch instead. I think that's something we could do reasonably easily. It would also be nice if this didn't just snapshot latest main, but instead took the last commit where all checks passed. I can look into how we would query this with the GitHub API.

@stellaraccident, since you created this snapshot. Can you comment on its stability and whether I could piggy back off this?

[stellaraccident]

This should be pretty easy. I'd recommend that we come up with a real version number scheme and start cutting periodic release branches (weekly?, monthly?, on demand?).

The release jobs are already broken in two and ready for this. All schedule_snapshot_release does is kick off the real job after creating a tag. You'll see that we feed it metadata like this:

inputs: '{"package_suffix": "-snapshot", "package_version": "${{ env.package_version }}", "release_id": "${{ steps.create_release.outputs.id }}"}'

If doing this manually for an "official" release, you could either build automation, or just manually create the branch/tag and fill in the values when kicking off a new Build Native Release job. For real releases, you would use the empty string for package_suffix, a real version number for package_version and a release_id as created from the releases API or interactively. Might take some amount of fiddling to get it right, but I think the automation is sound. Might want to read a bit about correlation between releases and versions. I think the release system is all tag based. Probably plenty of examples around for what we are trying to do though.

[stellaraccident]

(you should be able to try this interactively from the GitHub UI. It is all parameterized. Just use a separate package_suffix or experiments)

[stellaraccident]

Linux packages are stable. MacOS and Windows are WIP (and do not block if they fail).

[hcindyl]

Thanks for the pointers. GitHub UI doesn't seem to be able to create/update a branch, only tags.

In our specific case, we would like to track IREE closely with repo manifest, but might not be as close as to follow ToT. If the workflow can create the release- branch after updating snapshot tag, as well as merge to a up-to-date snapshot release branch, then the manifest only needs to point to the snapshot release branch without worrying about updating the release version number.

[stellaraccident]

I think I'm a little bit confused on exactly what would be helpful here. A branch per snapshot tag, or a snapshot branch with specific tags that keep moving forward, or (as I originally thought) a less frequent/better tested release? Might be easier to chat rt. Feel free to ping me on discord.

[GMNGeoffrey]

Cindy and I chatted for a bit yesterday (before I created this issue). I believe the desire is for an updating branch that points at a green and not-too-stale, but not quite as fast-updating as tot IREE. The greenness thing seems like it would be useful to incorporate into our snapshotting regardless. So I think the first desired action would be to push to a branch at the same time as setting a new snapshot tag, and the second would be to ensure that we schedule snapshots for a green commit.

[stellaraccident]

Sounds like the main thing we need is a way to query the latest green commit. Then update the snapshot workflow to reference that.

[hcindyl]

Sounds like the main thing we need is a way to query the latest green commit. Then update the snapshot workflow to reference that.

That would be helpful, but the original FR (and the first desired action in Geofferey's comment) is to push a branch (with a fixed branch name) in the snapshot workflow alongside the snapshot tag so our manifest could track.

[stellaraccident]

Ok. You do realize that there is nothing special about those commits without checking tests in some way: they are just whatever is the tot at the times of day the workflow kicks off.

[hcindyl]

Yes, I understand that's the current status, so the second step (an important one, I admit), is to make sure the workflow picks the latest green.

[GMNGeoffrey]

Yup I think nothing to complicated here. Just wanted to check in that you didn't have plans to redo the way the workflow was setup or whatever.

[hcindyl]

https://github.com/talentpair/last-green-commit-action may help to query the green. @GMNGeoffrey is this you can help take a look quickly?

[hcindyl]

The last green check is tracked at https://github.com/google/iree/issues/6314. For a snapshot branch, can we add something like this?

diff --git a/.github/workflows/schedule_snapshot_release.yml b/.github/workflows/schedule_snapshot_release.yml
index 47750fb0a..dc23485eb 100644
--- a/.github/workflows/schedule_snapshot_release.yml
+++ b/.github/workflows/schedule_snapshot_release.yml
@@ -35,6 +35,13 @@ jobs:
           branch: main
           tags: true

+      - name: Updating snapshot branch
+        uses: ad-m/github-push-action@v0.6.0
+        with:
+          github_token: ${{ secrets.WRITE_ACCESS_TOKEN }}
+          branch: snapshot
+          tags: false
+
       - name: Create Release
         id: create_release
         uses: actions/create-release@v1

This will help our branch bot to track the snapshot much easier (instead of tracking an ever-changing tag name). In addition, our team can then download the snapshot asset instead of building the host binaries from scratch for the cross-compilation. With more people in the team, it will tremendously save the development overhead.

[GMNGeoffrey]

Sure :-) Want to send a PR or did you want me to do it? It looks to me like what you've got is right and there's basically no good way to test GitHub actions so... #yolo?