Tag/Release Strategy using GitHub Actions

[ptheywood]

Releases are tagged commits with extra information (prerelease?, description, extra zip files etc) which can then be used with predictable download locations (for zip download).

For FLAMEGPU 1, when we create a (minor?) release we also manually attach a zip file containing the compiled executables for windows, using the visual studio and CUDA versions within the vcxproj files, so users do not have to compile code to try FLAME GPU.

For FLAME GPU 2 we can continue and automate this tradition (through github actions). We could also attach artifacts containing python related parts, or the library + headers.

GitHub Actions can be triggered for both push.tag events and release events.

When a new tag is pushed to github the on.push.tags event is triggered (can be filtered by tag name). This includes new tags created by creating new releases from untagged commits through the github releases UI.

Release events are assocaited with a given tag, either pre-existing or newly created through the UI. There are 6 types of release events. Actions only occur for non-draft releases.

• published - when an event is made non-draft
• unpublished - never acted upon, as unpublished === draft.
• created - when a new release is created and published at the same time (can be created but unpublished, so action does not trigger)
• edited - when the text of the release is modified, but not when the tag is changed (during testing)
• deleted - when a published release is deleted
• prereleased - when a published release is mared as a pre-release (creation or edit)
• released - when a published release is marked as not pre-release (creation or edit)

Release Proposal(s)

Option 1 - manual releases

• On tag events, trigger more thoroguh CI (all supported compute capabilities on all supported platforms) (if not already triggered for that commit?)
• On release events of the types published build artifacts, and attach to the release

New releases should always be created pre-release, and only marked as release once we are confident in the release (i.e. all tests + maybe manual tests).

Once we have GPU enabled actions (#97) we could automate/enforce the release/pre-release status.

• on pre-release events, run the gpu-enabled CI test suties - if it passes mark as a release
• on released events, if the mode detailed CI has not been ran, mark it as pre-release
  • This might require adding a PAT to result in the workflow-launched-by-workflow.
  • Or alternatively, on released events mark as pre-release then run the tests and if it passes mark it as a release again.

There is an edge case where if the tag which a release points to is changed, it does not trigger a release event - The solution is to not change release tags. Alternatively it may be possibly to run an action on push.tag events, which checks for an existing release associated with that commt hash via the api.

We may also need to restrict the author of the tag (not sure how PRs from forks with tags work, but we don't want to do this for external tags without explicit support)

Option 2 - automatic releases based on tags.

On tag events trigger the more thorough CI / actual tests. If it passes, create a new release based on the tag, and attach relevant artefacts. This would likely need the thorough build to include multiple jobs for the respective OS's being tested:

• Job 1 - linux, save artefacts
• Job 2 - windows, save artefacts
• Job 3 - create and the release and attach. Depends on Jobs 1 and 2 being successeful (for the full build matrices)

A mechanism for getting the changelog for the description would be required.

---

I've made this separate to #290 as it's lower priority / longer-term.

[ptheywood]

Two main Release Workflow options.

1. Tag-based.
2. Manual Release based.

Versioning.

At this point, we need to decide on versioning (exactly).

Current plan is to use SemVer.

I.e. FLAME GPU2 MAJOR.MINOR.PATCH[-PRE.RELEASE.STUFF]

If we're planning to publish to pypi, we must follow PEP440, which supports SEMVER, but also supports date based schemes as the alternative (i.e. 2012.4).

Assuming we follow semver, we need to take a little care with the python version number re: pre-release builds.

Build Matrix

• Multiple OS
  • Windows
  • Ubuntu
  • Docker Image creation?
• CUDA Version(s)
  • Single version? Multiple?
  • Implictly set the thorough CUDA_ARCH for this.
• Multiple configurations
  • Release, SEATBELTS=OFF
  • Release, SEATBELTS=ON
  • Debug, SEATBELTS=ON
• Visualisation Builds?
  • Do we want to provide pre-compiled visualisation builds?

So, the options are:

• SermVer Start at 0., for non-stable API
  • FLAME GPU 2 0.0.0-alpha / python 0.0.0a
• SermVer Start at 2., to avoid confusion with FLAME GPU 1.x
  • FLAME GPU 2 2.0.0-alpha / python 2.0.0a
• Date based. Probably not given release candence.
  • FLAME GPU 2 2021.07.

General Release workflow:

1. Create a pre-release version of some kind, then run some checks on it again / wait and see if things are happy.

Tag-based

1. On push of a tag matching the pattern vX.Y.Z[-P]) perform a thorough build (above), storing relevant files as archive.
2. Iff all these builds pass
   • create a release, autocomplete the message, attach artifacts. Mark as pre-release or release based on the tag number.
   • Push something to pypy. Maybe leave this as manual initially?

Pros:

• Very automated. Cons:
• maybe too automated
• following semver cannot retract/overwrite a bad release
• If a tag is force pushed, may create another release...
• Actions causing actions is a little fun to do
• Manually created releases with a new tag would likely break things.

Release-based

1. Push tag to github
2. Manually create a release via GitHub using that tag.
3. Tag-based thorough build, so we know if the tag is OK prior to release creation.
4. Publishing of the release triggers the thorough build. If it passes,

Alternatively, if the pushing of a tag to github is skipped and a new tag is created via the new release UI

1. Create new release for a new tag based on the current head of the main branch vi UI
2. Run thorough CI
3. Attach artifacts and mutate the release body?

Pros:

• Less automated
• If a tag is force pushed, the release binaries will not change Cons:
• Less automated
• If a tag is force pushed, the release binaries will not change
• Draft releases won't work right
• Probably can't enforce release creation to wait for tag CI, but that will be on us.
• Possibly longer time between tag push and binaries being uploaded.

Questions:

1. What exact versioning scheme do we want to use.
   • Major.Minor.Patch[-Prerelease]
     • Start at 0. for semver implications
     • Start at 2. to minimise confusion
   • YYYY.MM.patch
2. Binary Release Build Matrix
   • How many CUDA versions per binary release?
   • Visualisation? (possible, might be difficult)
   • Python - How large a matrix in general?
     • Still need to investigate the correct way to specify alternates. Initially a single release is probably OK? CUDA version might be very very important.
3. Tag-based or Release Based actions / workflow.
   • Do we want to manually create releases from pushed tags, once thorough CI has already ran?

@mondus @Robadob @MILeach

Todo:

• [ ] Resolve the questions above
• [ ] Embed the version number in the source rather than just relying on git hash
• [ ] Implement a better mechanism for external projects to pin against the main repo (and vice versa) via version numbers.
• [ ] Figure out the file structure for uploaded zips / instructions for users to download.
  • Do we just zip build and expect them to download it, or not
• [ ] Figure out python Can we attach the python wheel as an alterantive to pypi?
• [ ] Implement the github actions workflow(s) (in a fork) for
  • [ ] Thorough building
  • [ ] Attaching binaries to a release
  • [ ] Pypi

Associated Issues:

• 163

• 296

• 514

[ptheywood]

Personally, i'm in favour of:

1. v2.MINOR.PATCH-PRERELEASE
   • I.e. Start with 2.0.0-alpha.
   • 2.0.0 as the first non-pre release when we've finalised the initial API.
   • 2.1.0-rc1 mark release candidates for testing (and potentially not upload them to pypy?)
2. Matrix:
   • A single CUDA version per binary release seems fine to me, especially with recent CUDA's improvements in ABI stability (we just need to pick the correct version). We can change this later if needed.
   • Vis builds is probably worthwhile, if they work when extracted on random machines that may not be a perfect match. Might be an issue where dependencies are missing.
   • Initially keeping the python footprint small is probably simplest. Need to investigate more things really but it is time consuming.
3. Thorough build on Tag push, plus Binary build on manual Release creation is my preference, but both should be fine.

[Robadob]

1. What exact versioning scheme do we want to use. I agree that YYYY.MM.patch doesn't really make sense unless we want to commit to a regular release schedule (which I don't think we should). Less concerned about the 0 vs 2 thing.

2. Binary Release Build Matrix

• 2 major versions minimum, less concerned how many minor versions per major. Last release of previous major + all minor releases of current, e.g. past 5 minor releases might be the best compromise. But 5 seems like alot still.
• Vis where feasible, to me it seems reasonable to tell Linux users to build themselves if they want vis if there are going to be compat issues. Not sure how viable vis inside docker would be, that might work better (e.g. only binary release with vis is the docker release).
• Python, release +-SEATBELTS with a single CUDA version is all that needs to be offered imo, with us encouraging most users to use docker for convenience.

3. Tag-based or Release Based actions / workflow.

• Automated would always be preferable, if we can get it to work as desired.

[ptheywood]

Meeting decisons:

• Manual Release based workflow (with thoroguh build on semver tag push)
• SemVer Start at 2.0.0-alpha (or maybe 2.0.0-pr1)
• For now, Attach .whl's to the release for manual install. Pypi / Conda to come later.
  • Pypi whl filesize limit of 60MB is already being exceeded, and it will only get bigger.
• Only VISUALSATION=ON builds.
• Single CUDA version for binary releases. Fat binary for all supported architectures
• 3 Python versions?
• Python wheel filenames need to factor in all the important bits
• Release, Debug and Seatbelts off wheels
• C++ binary releases with the lib. For now this might not be super useful (as the lib, include and cmake dirs are all needed for this to be useful)
• Installation instructions on the readme + documentation will need updating.

Will need install instructions on the docs, via wheel / zip download