Release automation

[cameel]

There are multiple things we would like to automate in the release process to reduce the amount of error-prone manual steps on the release checklist. Here's what we could do:

Must have

• [ ] Creating a PR with binaries in solc-bin.
  • Instead of having to manually submit binaries to solc-bin, we could have c_release_binaries (or a new dependent job) run the update script and create a PR.
  • Preferably this would not involve giving the CI direct access to solc-bin. To achieve that it would be best if we had a workflow in solc-bin that would be triggered when the binaries in the main repo are ready (e.g. via workflow_dispatch) and that workflow would pull them in.
• [ ] Publishing binaries on the github release page.
  • c_release_binaries could check if a page for the new release exists. If it does, it would upload them.
    • It should gracefully handle being restarted - in case we forget to create the page beforehand.
  • The releases are editable so this does not need a human in the loop. If something goes wrong we can always manually replace the binaries with correct ones.
• [ ] Creating PRs to update Homebrew formulas.
  • This should be triggered by the release being published on github.
• [ ] Release verification script.
  • A script that accepts the version number and ensures that the release is complete (i.e. things are up to date and all relevant artifacts exist and are publicly available):
    • Changelog is pointing at the next, unreleased version and has no gaps in version numbers.
    • Bug list contains the latest version.
    • Tags corresponding to published versions exist.
    • Github release has been published and the warning removed.
    • Github release page and solc-bin contain binaries and their hashes match the file list in solc-bin.
    • Binaries from solc-bin have been successfully mirrored to https://binaries.soliditylang.org and hashes match.
    • Github release page contains the source archive.
    • Homebrew formulas have been published.
    • Docker image has been published.
    • PPA packages have been published.
    • solc-js has been published.
    • solc-js version has been bumped and the release was tagged.
    • The documentation has been published.
    • Blog posts are published and do not contain broken links.
    • Release information at soliditylang.org is up to date.
    • (Optional) Report if there's anything that looks like an announcement on Reddit, Twitter and forum. This might require some heuristics and won't be foolproof.
  • The script should mainly check if things are published and available. It may check if they work but only in cases where it can't be easily delegated to CI.
  • The script should not fail when the release is incomplete. It should report which things are done and which are not.

Nice to have

• [ ] Script for updating version and changelog
  • Both scripts commit all changes and create branches. Optionally might also use Github CLI to actually submit a PR. Merging and review of the PR is expected to still be manual.
  • Creating a PR that finalizes the current version
    • Sorts the changelog and removes empty sections.
    • Replaces (unreleased) with the current date next to the version at the top of the changelog.
    • Updates the bug list.
  • Creating a PR that initializes a new version
    • Bumps version in CMakeLists.txt
      • Can take a parameter saying which component to bump: major/minor/patch.
    • Creates a new section for the next version in the changelog.
• [ ] Preparing a source archive and publishing it on the github release page.
  • Only if it's very simple and #9720 is not done by then.
• [ ] Script for preparing release notes
  • Prints release notes template with filled-out changelog and the list of contributors.
  • Optionally, creates a draft release using Github CLI.
• [ ] Is there anything in the solc-js publication process that makes sense to automate?

[ekpyron]

I'd say we should generally prefer https://github.com/ethereum/solidity/issues/9720 over automating preparing the source archive.

As for Publishing binaries on the github release page. we should check what permissions this requires (it's probably at least repo write) and actually also whether we can restrict current permissions (i.e. make it harder to just replace binaries on finalized past releases).

[cameel]

I'd say we should generally prefer #9720 over automating preparing the source archive.

Well, just running the script to build seems simple enough. Might be worth doing it first anyway, just to automate that fully.

As for Publishing binaries on the github release page. we should check what permissions this requires (it's probably at least repo write) and actually also whether we can restrict current permissions (i.e. make it harder to just replace binaries on finalized past releases).

We should probably do it in github actions and just use the default token. Then we can make all tokens readonly by default and use the permissions keys in the action to explicitly only request that one permissions.

[sambacha]

Is it possible to also add nixpkgs to the release process?

e.g. Dapptools uses this release process https://github.com/dapphub/dapptools/blob/master/nix/make-solc-static.sh - the binaries need to be patched to include additional ELF information, this is also a requirement for solc to work with foundry if used via nix. see this note https://github.com/shazow/foundry.nix#nixos-caveat

I can open a new issue if needed.

edit: this effects foundry as well https://github.com/foundry-rs/foundry/issues/545

[sambacha]

Any update on the nix related comment I have raised, I can get this issue done if you will consider a PR that I submit?

[EvanPiro]

I'm also eager to have this addressed to unblock nixpkgs support and am willing to assist in the effort.

[cameel]

@sambacha What exactly would that involve? Running that script to update solc-static-versions.nix and making a PR or actually rebuilding the binary? Currently going through the whole release checklist already takes more than we'd like and we're looking for ways to reduce it. So I'd rather not add another manual step. However, we're already doing similar things in some cases (Homebrew) so if it could be automated from the start and not make the release longer, I guess it could be ok.

Maybe it would be enough for us to just trigger some CI job on your side when the release is published so that you can do whatever other steps are required on your side? For example we could have a job on our side that is triggered by the release published event that would ping your CI, e.g. via workflow_call if you're using Gihub Actions. We'd accept a PR implementing that.

I can open a new issue if needed.

Yeah, I think it would be better to open a new issue to track this. This one is really about automating what we already have.

[sambacha]

Thanks @cameel for replying, let me coordinate with a few other people that are interested in this. I will create a new issue for it.

Will get back to you later this weekend, early next week.

Cheers

[sambacha]

@sambacha What exactly would that involve? Running that script to update solc-static-versions.nix and making a PR or actually rebuilding the binary? Currently going through the whole release checklist already takes more than we'd like and we're looking for ways to reduce it. So I'd rather not add another manual step. However, we're already doing similar things in some cases (Homebrew) so if it could be automated from the start and not make the release longer, I guess it could be ok.

Maybe it would be enough for us to just trigger some CI job on your side when the release is published so that you can do whatever other steps are required on your side? For example we could have a job on our side that is triggered by the release published event that would ping your CI, e.g. via workflow_call if you're using Gihub Actions. We'd accept a PR implementing that.

I can open a new issue if needed.

Yeah, I think it would be better to open a new issue to track this. This one is really about automating what we already have.

This will actually be easier now to accomplish, at least for NixOS

"Nix-LD-RS provides a shim layer that allows users to specify the necessary libraries for each executable and improves the user experience by allowing users to easily run binaries from third-party sources and proprietary software"

https://github.com/Mic92/nix-ld this will become the default for patching in the future, so any changes potentially would only be needed in the short run