How do we push changes in this repo downstream to our communities?

[sgibson91]

Context

This is a template repo that many of our communities use to self-serve images for their hubs. We often make changes and improvements to this template, e.g., #10. But it's not clear to me how we push those improvements to the repositories already using this template.

Since this is a template repo, not a fork, trying to pull changes from this repo into a downstream repo in the "usual" way (say, by adding this repo as a template and running git fetch --all && git merge...) will result in a trying to merge unrelated commit histories error.

Proposal

No response

Updates and actions

No response

[damianavila]

It seems allowing unrelated commit histories could make the trick?

--allow-unrelated-histories

Source: https://stackoverflow.com/questions/56577184/github-pull-changes-from-a-template-repository/56577320#56577320

We definitely need a good story here for downstream fixes... related discussion https://github.com/2i2c-org/infrastructure/issues/1145#issuecomment-1088031412

[yuvipanda]

I think we should move docs elsewhere to a more centralized place and link to it. Pushing git changes out is unsustainable especially if communities make downstream changes.

[sgibson91]

@yuvipanda what about changes to workflow files?

[yuvipanda]

Good question! So I think we can separate the contents of this repo into three parts:

1. Docs - this we should centralize, and link to from here.
2. CI / CD related things - we might need to make mass changes to these, primarily to 'push' changes here downstream.
3. Actual environment specifications - this will be custom for each repo, but we might still need to make mass changes to this.

We'd definitely have to write some automation scripts to deal with (2) and (3), but I don't know exactly how they'll look until we first do them. I think git merge is probably going to be too complicated given there are probably unrelated downstream changes, so I'd suggest our automation script is more specific and uses either diff and am directly.

Given that we've had to do #10, I think a plan of action is to:

1. Centralize the docs now
2. Work on some scripts to update a given list of repos in a very specific way. We can do this to replace all READMEs with a link to our centralized docs now as well?

I think this can become a bit of a deep hole with diminishing returns, so I want us to be slightly careful :)

[yuvipanda]

As a proof of concept for what I mean by diff and am,

1. I made https://github.com/2i2c-org/hub-user-image-template/pull/13 to test a change to the github workflow files.
2. On a local checkout of https://github.com/2i2c-org/utoronto-image/pull/30 (as an example) I ran curl -L https://github.com/2i2c-org/hub-user-image-template/pull/13.patch | git am. Note the .patch appended to the PR URL - it provides a patch file that can be applied using git am. This is one of the many steps git merge does internally, but git exposes all this to us too. This lets us apply only specific PRs to a bunch of repos without having to worry about merge histories. Regular conflict resolution can be performed when the patch doesn't apply cleanly
3. Open a PR like this: https://github.com/2i2c-org/utoronto-image/pull/30. We can potentially automerge them too in our automation.

This would let us port a set of changes across repos without having to deal with all the extra bits that git merge does.

[sgibson91]

I really like this proposal!

[damianavila]

I really like this proposal!

Indeed! I have successfully used this patch-based approach in the past 😉.

For step 2, are you envisioning some action in the child repo that triggers based on detected changes in the parent repo and do all the curl and git am dance... and finally opens a PR (potentially including automerge)?

[sgibson91]

I had another thought on this. With regards to the workflow files themselves, we could transform them into reusable workflows such that everyone downstream would benefit from updates, while allowing us to centrally manage them.