Workflow for updating from official repo

[jwodder]

Closes #1.

@yarikoptic Please comment if you want the branch the workflow is on to be adjusted.

[jwodder]

@yarikoptic I've amended the workflow to pull into upstream/master and then merge that into master. I've also created the upstream/master branch on this repo.

Something to keep in mind that I forgot to mention: New commits added to the repository in this way won't cause any workflows to run unless we use a personal access token.

[yarikoptic]

oh, I had no idea. uff. Since I have already added a DATALAD_GITHUB_TOKEN secret which is I guess "my" personal access token, we could use it here?

Also I totally forgot the desire to trigger workflows for releases of git-annex! (e.g. ref https://github.com/datalad/datalad-extensions/issues/30)... First I thought that we could just detect freshly fetched tags (between master...upstream/master) and then merge them explicitly and push explicitly with DATALAD_GITHUB_TOKEN to trigger workflow on them. But not sure if that is a good idea. It would miss rare (but happened in the past, e.g. 7.20200309) releases from release branches... We could have create per-release branches and this way ensured that we do not miss any tag (for which we do not have a branch) but that sounds like an overkill as well.

Overall the idea of "merged" git-annex tree becomes less and less appealing - binds workflows and git-annex too much :-/

Let's think about it a bit more: I see lots of ways how workflows could be triggered (documented on the same page you referenced). I wonder if without any kind of a bot we could still have workflows run and report back on PRs against upstream/master even if that PR branch would not contain .github/workflows? If it is possible, then we probably could just establish master with workflows and configuration on when/how they should be triggered. We would then not merge upstream/master into master, and have workflows read-tree explicitly specified commitish (be it some commit pushed to upstream/master or some release tag) or PR state to build git-annex. Build artifacts upload would still need to happen to outside of this repository/packages etc, so it seems we do not even to worry about master not reflecting "what we have already built".

But if that would be the same workflow which works on upstream/master, and release tags, and PRs - AFAIK there would be no way to get a dedicated banner for README here or in datalad-extensions to report "health status" of git-annex. So, most likely we would need to template (copy) the workflows and instantiate/trigger dedicated ones for specific occasion(s) to reflect the state for a dedicated event.

WDYT @jwodder ?

[jwodder]

In order for a GitHub Actions workflow to trigger automatically on a PR, the workflow YAML file has to exist at the Git ref(s) involved in the PR; i.e., it has to be either already in the base branch or in the PR branch. (See the list under "The following steps occur to trigger a workflow run:" on the linked workflow triggers page.) If we're considering a setup where workflows are in one branch and the source code is in another, PRs made to the source code branch won't trigger any workflows directly.

As for triggering workflows indirectly, the following options come to mind:

• We set up a server written in Flask or the like which the repository sends webhooks to on PR events, and then the server triggers webhooks using the REST API.
• We set up a workflow that runs on a schedule (The most often a scheduled workflow can run is every 5 minutes), checks for new PR events since the last run, and triggers the appropriate workflows via the REST API.

Note that, for the indirect setups, I don't think it'd actually be necessary to even have the workflows in the same repository as the code, so we could go back to keeping the workflows in datalad-extensions.

[yarikoptic]

I think that

We set up a workflow that runs on a schedule (The most often a scheduled workflow can run is every 5 minutes), checks for new PR events since the last run, and triggers the appropriate workflows via the REST API.

But would it report status back to those PRs either tests passed or not? (never tried). If it would -- would be the smoothest/easiest way to do ATM. Could you try?

Note that, for the indirect setups, I don't think it'd actually be necessary to even have the workflows in the same repository as the code, so we could go back to keeping the workflows in datalad-extensions.

I think it would be best to collect everything git-annex building/testing related (not extensions related) here, and keep datalad-extensions specifically for extensions AND only displaying the health status for primary datalad repo and git-annex (here).

[jwodder]

But would it report status back to those PRs either tests passed or not?

That can be done via the checks API, but it appears we'd need to set up a GitHub App rather than just a workflow for that. Alternatively, we could create a robot GitHub user and use that account to post comments on the PRs.

[yarikoptic]

ok, we can do that (reports back on PRs on code) when immediate need comes. If we could at least submit workflows (without reporting back to PRs) - that would be already great. So let's proceed with having local master to be just our workflows/helpers and keep git-annex source code intact in upstream/master.

[jwodder]

@yarikoptic I've adjusted the workflow to only update upstream/master. Should I go ahead and set master to just a single commit with an empty file?

[yarikoptic]

Yeap, You can even initiate it with README.md describing the "setup" (where to find git-annex branches) and that master only to contain workflows and helper tools.

[jwodder]

GitHub seems to have automatically closed the PR when I force-pushed master, and the "Reopen and comment" button is greyed out with the message "The gh-1 branch was force-pushed or recreated." I'm going to submit a new PR.

New PR: #6