[Feature]: Adding additional keys to upgrade.proto

effofxprime commented 1 year ago

Summary

In ref. to: https://github.com/cosmos/cosmos-sdk/pull/16956 @julienrbrt

Currently, there is no defined place to put the release version tag and/or commit hash for Upgrade plans.

Adding this information is beneficial in a few ways.

Provides an unequivocal field for the version tag and commit hash, eliminating any confusion or issues when using git to fetch the update
Will provide a back history for later reference
Will allow admins to create tools to verify upgrade versions, as well as automatically fetching them.

If approved to modify and add the additional keys, backporting to versions, 0.45.x, 0.46.x, etc.. so all current running chains can benefit from it.

Problem Definition

I have been working on an upgrade script that I plan on releasing for all to use. One of the issues I am currently running into is parsing the information found in Upgrade Gov Props/Plans. There is really no standard for listing the version/commit has to upgrade to. For example, Stargaze's recent upgrade listed the upgrade version as v11 but the tag is really v11.0.0. Another issue that can arise is, when multiple versions are listed in the description, like what else the upgrade is updating to, determining the proper version tag for GitHub is not reliable.

I believe this will help many with upgrades. There will be no question what the release tag and commit hash should be when verifying your version. Especially for those who compile their own binaries.

Proposal

Updating proto/cosmos/upgrade/v1beta1/upgrade.proto to include keys to define the proper github release tag and corresponding commit hash.

Example:

  string github_release_tag = 2;

  string github_release_commit_hash = 3;

julienrbrt commented 1 year ago

Usually, plans contain a binary url as well: https://www.mintscan.io/umee/proposals/91, so you could assume a tag from there. However, this is correct that it may not be the most reliable.

I think adding a commit is useful (we could add a feature in cosmovisor to support that too); however, I do think the fields should not be GitHub-specific but just a directly clonable link instead.

effofxprime commented 1 year ago

I've not seen binary url's for any of the upgrade plans I've seen so far. I do have limited experience and only validate for a handful of Blockchains. I was recently made aware of the entry though after speaking with Reece from Juno and Raul from Bitcanna.

Also, I know that some blockchains have different binary versions, ARM, AMD, WINDOWS... and when there are multiple, listing one doesn't cover everyone/every case.

To be more general, the fields could be named:

repo_release_version_tag = X

repo_commit_hash = Y

In the event someone chooses a service outside of Github, it wont be confusing.

I do think that both fields would be beneficial. The release version tag will instantly tell you the actual expected binary version. The commit hash will give you an unequivocal method for checking out the branch without worry that your local repo failed to properly update tags. I have had a single instance with Chihuahua when upgrading, my checkout with the tag didn't pull the proper latest commit despite having done git fetch --all.

When using Github's API, I can get the download links for the precompiled binary with the version tag.

Example here: curl -sL https://api.github.com/repos/JackalLabs/canine-chain/releases | jq -r '.'

This will give the entire list of all releases.
To get the latest release curl -sL https://api.github.com/repos/JackalLabs/canine-chain/releases/latest | jq -r '.tag_name'

The problem with trying to use this in the assumption it is the expected version for the upgrade is that, some chains will wait until after the upgrade to post the new version as a release. This is to avoid new users who are trying to start a node with a new binary version that has yet to be implemented on chain.

If we have the tag name, then I can use the API to filter results so I know the exact link for the pre-compiled binary. On top of knowing the tag name/commit hash, using a script to auto-compile your local repo to build your binary could be automated for you. Returning a list of all current release version tags curl -sL https://api.github.com/repos/JackalLabs/canine-chain/releases | jq -r '.[].tag_name'

And if there was an exact method on chain to grab the proper tag, then we can get the info we're after. In the example for Jackal here, lets use v2.1.0 curl -sL https://api.github.com/repos/JackalLabs/canine-chain/releases | jq -r 'first(.[] | select(.tag_name == "v2.1.0" ))'

This returns

{
  "url": "https://api.github.com/repos/JackalLabs/canine-chain/releases/111297890",
  "assets_url": "https://api.github.com/repos/JackalLabs/canine-chain/releases/111297890/assets",
  "upload_url": "https://uploads.github.com/repos/JackalLabs/canine-chain/releases/111297890/assets{?name,label}",
  "html_url": "https://github.com/JackalLabs/canine-chain/releases/tag/v2.1.0",
  "id": 111297890,
  "author": {
    "login": "TheMarstonConnell",
    "id": 34043723,
    "node_id": "MDQ6VXNlcjM0MDQzNzIz",
    "avatar_url": "https://avatars.githubusercontent.com/u/34043723?v=4",
    "gravatar_id": "",
    "url": "https://api.github.com/users/TheMarstonConnell",
    "html_url": "https://github.com/TheMarstonConnell",
    "followers_url": "https://api.github.com/users/TheMarstonConnell/followers",
    "following_url": "https://api.github.com/users/TheMarstonConnell/following{/other_user}",
    "gists_url": "https://api.github.com/users/TheMarstonConnell/gists{/gist_id}",
    "starred_url": "https://api.github.com/users/TheMarstonConnell/starred{/owner}{/repo}",
    "subscriptions_url": "https://api.github.com/users/TheMarstonConnell/subscriptions",
    "organizations_url": "https://api.github.com/users/TheMarstonConnell/orgs",
    "repos_url": "https://api.github.com/users/TheMarstonConnell/repos",
    "events_url": "https://api.github.com/users/TheMarstonConnell/events{/privacy}",
    "received_events_url": "https://api.github.com/users/TheMarstonConnell/received_events",
    "type": "User",
    "site_admin": false
  },
  "node_id": "RE_kwDOHbW_bM4GokVi",
  "tag_name": "v2.1.0",
  "target_commitish": "v2.x.x",
  "name": "v2.1.0",
  "draft": false,
  "prerelease": false,
  "created_at": "2023-07-06T03:25:09Z",
  "published_at": "2023-07-06T17:30:18Z",
  "assets": [
    {
      "url": "https://api.github.com/repos/JackalLabs/canine-chain/releases/assets/115861277",
      "id": 115861277,
      "node_id": "RA_kwDOHbW_bM4G5-cd",
      "name": "canined-Linux",
      "label": "",
      "uploader": {
        "login": "github-actions[bot]",
        "id": 41898282,
        "node_id": "MDM6Qm90NDE4OTgyODI=",
        "avatar_url": "https://avatars.githubusercontent.com/in/15368?v=4",
        "gravatar_id": "",
        "url": "https://api.github.com/users/github-actions%5Bbot%5D",
        "html_url": "https://github.com/apps/github-actions",
        "followers_url": "https://api.github.com/users/github-actions%5Bbot%5D/followers",
        "following_url": "https://api.github.com/users/github-actions%5Bbot%5D/following{/other_user}",
        "gists_url": "https://api.github.com/users/github-actions%5Bbot%5D/gists{/gist_id}",
        "starred_url": "https://api.github.com/users/github-actions%5Bbot%5D/starred{/owner}{/repo}",
        "subscriptions_url": "https://api.github.com/users/github-actions%5Bbot%5D/subscriptions",
        "organizations_url": "https://api.github.com/users/github-actions%5Bbot%5D/orgs",
        "repos_url": "https://api.github.com/users/github-actions%5Bbot%5D/repos",
        "events_url": "https://api.github.com/users/github-actions%5Bbot%5D/events{/privacy}",
        "received_events_url": "https://api.github.com/users/github-actions%5Bbot%5D/received_events",
        "type": "Bot",
        "site_admin": false
      },
      "content_type": "application/octet-stream",
      "state": "uploaded",
      "size": 77895568,
      "download_count": 21,
      "created_at": "2023-07-06T17:35:22Z",
      "updated_at": "2023-07-06T17:35:25Z",
      "browser_download_url": "https://github.com/JackalLabs/canine-chain/releases/download/v2.1.0/canined-Linux"
    },
    {
      "url": "https://api.github.com/repos/JackalLabs/canine-chain/releases/assets/115861276",
      "id": 115861276,
      "node_id": "RA_kwDOHbW_bM4G5-cc",
      "name": "canined-macOS",
      "label": "",
      "uploader": {
        "login": "github-actions[bot]",
        "id": 41898282,
        "node_id": "MDM6Qm90NDE4OTgyODI=",
        "avatar_url": "https://avatars.githubusercontent.com/in/15368?v=4",
        "gravatar_id": "",
        "url": "https://api.github.com/users/github-actions%5Bbot%5D",
        "html_url": "https://github.com/apps/github-actions",
        "followers_url": "https://api.github.com/users/github-actions%5Bbot%5D/followers",
        "following_url": "https://api.github.com/users/github-actions%5Bbot%5D/following{/other_user}",
        "gists_url": "https://api.github.com/users/github-actions%5Bbot%5D/gists{/gist_id}",
        "starred_url": "https://api.github.com/users/github-actions%5Bbot%5D/starred{/owner}{/repo}",
        "subscriptions_url": "https://api.github.com/users/github-actions%5Bbot%5D/subscriptions",
        "organizations_url": "https://api.github.com/users/github-actions%5Bbot%5D/orgs",
        "repos_url": "https://api.github.com/users/github-actions%5Bbot%5D/repos",
        "events_url": "https://api.github.com/users/github-actions%5Bbot%5D/events{/privacy}",
        "received_events_url": "https://api.github.com/users/github-actions%5Bbot%5D/received_events",
        "type": "Bot",
        "site_admin": false
      },
      "content_type": "application/octet-stream",
      "state": "uploaded",
      "size": 79791448,
      "download_count": 0,
      "created_at": "2023-07-06T17:35:22Z",
      "updated_at": "2023-07-06T17:35:25Z",
      "browser_download_url": "https://github.com/JackalLabs/canine-chain/releases/download/v2.1.0/canined-macOS"
    }
  ],
  "tarball_url": "https://api.github.com/repos/JackalLabs/canine-chain/tarball/v2.1.0",
  "zipball_url": "https://api.github.com/repos/JackalLabs/canine-chain/zipball/v2.1.0",
  "body": "https://forums.jackalprotocol.com/t/software-upgrade-adjust-storage-provider-rewards/42\r\n\r\n## What's Changed\r\n* Feat: Mint Module Adjustments by @TheMarstonConnell in https://github.com/JackalLabs/canine-chain/pull/355\r\n* Feat: Provider Rewards to Zero by @TheMarstonConnell in https://github.com/JackalLabs/canine-chain/pull/358\r\n\r\n\r\n**Full Changelog**: https://github.com/JackalLabs/canine-chain/compare/v2.0.2...v2.1.0",
  "mentions_count": 1
}

You can see there is no commit hash here. But I am sure there is a query that you could start with the hash and work backward to get a version tag. I haven't looked into it/tried yet.

Taking a linux release into consideration here first, there are only a few terms that get used for pre-compiled binary releases.

linux
amd/amd64
arm/arm64 And others include
darwin
maxos
windows

My point here is that, I don't feel it's safe to assume parsing a version number from the Prop/Plan data, but I do think it's safe to assume that linux releases will have one of those 3 options in the file name. Leading to be able to get the download URL for the pre-compiled binary from the key browser_download_url

Long story short, I see having both making things easier.

Another thought is to add a couple more keys than the two suggested above. You mentioned about giving a direct link, what if there were also keys like:

upgrade_binary_source = LINK
upgrade_binary_{linux,amd}64 = LINK   #Could use linux or amd)
upgrade_binary_macos64 = LINK
upgrade_binary_arm64 = LINK
upgrade_binary_windows = LINK

Or something similar along those lines?

cosmos / cosmos-sdk