Reimplement CI/CD and release system

[Derpius]

Context

• We have a CI/CD pipeline which runs when a git tag is pushed matching v*.*.*
• The release artefact that's uploaded is generated on the fly using a python script in the scripts folder
• The GitHub workflow does not perform any caching of dependencies or cmake intermediate output
• The manual release process leads to audience-ready changes being delayed for no reason
  • There is no formal testing process before release, so manual releases make no sense
  • If we wanted testing, it should be done as a QA step before merging PRs

Scope

• Create a /release directory in the root of the repository
• Create the relevant static release files in that folder (INSTALL.md, gelly-bootstrap, etc.)
• Update the relevant CMakeLists.txt to set the binary output directory to /release/garrysmod/lua/bin (see VisTrace for an example of this)
• Update the relevant CMakeLists.txt to copy the flex dependencies (and any other dlls) to /release
• Update the root .gitignore to prevent staging the release binaries within the /release folder
• Create a new GitHub workflow which does the following:
  • Runs on any pushes to the master branch only
  • Checks out CMake
  • Checks out the repository with all submodules - TODO: Investigate caching the submodules to prevent additional clones
  • Restores a previous CMake build cache if available - TODO: Investigate 3rd party actions for CMake caching
  • Builds with a release preset - Future work: Use a build matrix as used by VisTrace to run the job for all profiles
  • Updates CMake build cache - TODO: Investigate 3rd party actions for CMake caching
  • If #61 is not complete:
  • Clones the sp-addon branch to a temp folder
  • Compile the addon with gmad.exe and output to /release/garrysmod/data/...
  • This is currently committed to the repository
  • Could: Remove gmad.exe from the repository and fetch it in the action
    • Open source here, but they don't seem to have release artefacts we could use. Might need to host ourselves or use steamcmd. See what Vurv did for wiremod's workshop CI/CD
  • Publish a release with that artefact using the version and body parsed in #56
• Delete the old workflow

Acceptance Criteria

• GitHub workflow has a significant performance boost from caching dependencies and build files where possible
• Python script is removed to reduce build complexity and pipeline runtime
• Workflow is run on every commit to the master branch

Other

This will be harder to do without #61

This depends on #56 (probably easier to do in parallel)

[yogwoggf]

Yeah this is probably one of the most important things to do, anyways Vurv implemented GMA packing by writing his own GMA implementation, but that has a lot of issues like the blacklist being out of sync with Facepunch's blacklist...

[Derpius]

Could either host gmad.exe ourselves or compile from source (and cache)

I would strongly suggest the former - Ideally not committed into the repo, but I can't think of any great alternatives

[yogwoggf]

All blockers done, will start working on this

[yogwoggf]

Nearly complete in branch 63-cd-ci, although I have yet to write logic for caching. It's a little difficult, every submodule is in wildly different places, not sure how to cache those. Although, it should be simple enough to cache previous builds.

[Derpius]

Two options for caching submodules:

Key as repository name/url

1. For each submodule, attempt to restore from cache with the repository name/URL
2. On cache hit, update the submodule (pull any changes)
3. On cache miss, init the submodule
4. Save the updated submodule to the cache

This is probably the fastest approach. You perform incremental updates whenever you update the submodule’s commit in the project. It probably doesn’t handle switching the branch of a submodule though.

Key as repository + commit SHA

1. For each submodule, attempt to restore from cache with the repository name/URL and commit hash
   • Retrieve target hash (not current) as explained here: https://stackoverflow.com/questions/20655073/how-to-see-which-commit-a-git-submodule-points-at
   • This may require the submodule to already be initialised. Can’t find an obvious way to grab the target hash otherwise (maybe can read directly from the parent’s .git)
2. On cache hit, do nothing
3. On cache miss, init the submodule and add the new cache entry with the commit hash

This is robust to changing branches, and is marginally faster on cache hit (but much slower on cache miss in the case where the repo was previously cloned)

[yogwoggf]

Caching for submodules is done, although not really done that well but it should do. Now I'm working on using sccache for caching, it's a "modern" ccache developed by Mozilla which has real nice plug-n-play support for everything (msvc, clang, gcc, etc).

[yogwoggf]

Sccache plugged in, but further blockers remain to switching to ubuntu cross-compilation. Will make a ticket

[yogwoggf]

All done in #70