Closed posvyatokum closed 7 months ago
1.37.0-rc.1 is planned to be released on 2024-01-23. See timeline above.
if you are tagged in a comment here, please respond. ๐ or ๐ reaction to the comment is enough. If you feel like you don't have any thoughts about the release, give a ๐ anyway (as in "I don't have any objections").
@gmilescu @akhi3030 @walnut-the-cat @wacban @telezhnaya
edited
I think we should also include #10450.
I think it would also be nice to include #10437 but please consider it a wish and not a requirement.
What is the date when all the tooling should be updated for new testnet version? e.g. near-cli 23/29/31 of January?
What is the date when all the tooling should be updated for new testnet version? e.g. near-cli 23/29/31 of January?
@telezhnaya If the tooling depends on protocol version (which does not seem true) then 31 of Jan. If it depends on release version of our testnet nodes, it should be updated 24-25 Jan, as it is approximately the date when Pagoda SRE's will update all our nodes.
other than what @wacban shared, no comments from my end. good to go
Status: We are finishing up the mocknet testing of resharding*. If everything goes right, we will release 1.37.0 on Tuesday 2024-02-27 around 18:00:00 UTC. If something goes wrong, we obviously will not.
@gmilescu @akhi3030 @walnut-the-cat @wacban @telezhnaya @khorolets @marcelo-gonzalez
If you are tagged in a comment here, please respond. ๐ or ๐ reaction to the comment is enough. If you feel like you don't have any thoughts about the release, give a ๐ anyway (as in "I don't have any objections").
* Our tests include:
I will post testing conclusions in this issue tomorrow. @marcelo-gonzalez please, do the same when you are finished with testing.
@posvyatokum I posted a longer message on zulip that I tagged people in, but it looks like there's a bug where validators can see different state roots under certain conditions, so I think that needs to be investigated and fixed before we can safely proceed
I also need to add one PR before we release 1.37
I need to change the logic of broadcast_tx_commit
back
https://github.com/near/nearcore/pull/9644#issuecomment-1965515825
Ok, then we are moving 1.37.0 release one week. Another issue that we will address during this week is legacy archival nodes. We don't think our mainnet legacy archival nodes will be able to be in sync with the chain in time for resharding, their performance is barely faster than the chain itself, so going through resharding is also questionable. We will do an announcement about deprecation of legacy archival nodes ASAP, so that validators have at least a week to migrate to split storage.
This is the commit we need to cherry pick https://github.com/near/nearcore/pull/10655
Status: We are running one final test of the whole 1.37.0 release. If everything goes right, we will release 1.37.0 on Tuesday 2024-03-05 around 18:00:00 UTC. If something goes wrong, we obviously will not.
@gmilescu @akhi3030 @walnut-the-cat @wacban @telezhnaya @khorolets @marcelo-gonzalez
If you are tagged in a comment here, please respond. ๐ or ๐ reaction to the comment is enough. If you feel like you don't have any thoughts about the release, give a ๐ anyway (as in "I don't have any objections").
Addressing previous issues:
@posvyatokum @nagisa @Ekleog-NEAR @khorolets Is the crates publishing also on track?
I donโt see any run of the crates publishing workflow. @posvyatokum as it seems youโre the release manager, did you know of the recent-ish (a few months, one release ago) changes to the release process that added it?
@Ekleog-NEAR semi-aware, definitely didn't see confluence update. I will add this step to the release template that we keep in github. Is this run correct/sufficient?
I will add this step to the release template that we keep in github
Thank you! I wasnโt aware of the existence of a release template on github, I guess there was probably a race condition between the migration from confluence to github and my adding it to confluence around late December. Please let me know if I can help updating any other process document to make this a smoother experience!
Is this run correct/sufficient?
No. The process documented on confluence is:
After cutting the release and creating the new branch, the release owner needs to publish the workspace-wide crates, that are versioned alongside neard. The process is:
Bump the workspace.metadata.workspaces.version field in the workspace Cargo.toml on the release branch to perform the publishing, and on the master branch to record the latest published version. Considering we are not careful about backward compatibility of internal crates, you should usually bump the major version of our current 0.major.minor versioning scheme, unless you manually checked the changes.
Run the appropriate workflow with, as parameter, the release branch or tag
In the release notes for the new neard version, record the corresponding version of the published nearcore crates, with a sentence like:
Synchronously released crates corresponding to this nearcore version were published with version ${{workspace.metadata.workspaces.version}}
Looking at the release branch, my guess is youโre missing the first step here, of bumping to 0.21.0
@Ekleog-NEAR I will reach out to you to check that we are doing everything right with 1.38 But also I need help with figuring out what we do now. Options in increasing order of my personal pain and mainnet risk from resharding perspective
I would also like to understand potential risks more. You can just point me to some old Zulip thread.
Nothing with 1.37, do everything right in 1.38
New crates matching 1.37 API must be published. Otherwise, we cannot support the latest RPC API changes in our Rust tooling.
I don't think you have to make a new binary release after you bump the crate versions.
@posvyatokum Iโd say:
Probably ignore this paragraph: If the 1.37 branch is supposed to be freezed (I seemed to understand we had switched to a tag-based release management scheme, but just in case weโre still on the old branch-based scheme), then any branch would do the trick and you could just run the workflow off the commit hash, or creating a new branch just for it (like the crates-0.20.x branch that we created between two releases, making a crates-0.21.x branch thatโd be just 1.37 + the version-changing commit)
As for the risks, @frol already exposed them just above :)
@frol All the crates should have been published as 0.21 :)
I believe it is time to close this issue. Feel free to re-open if you believe otherwise
This issue is for keeping history of the release.
Actual Timeline:
Wed 2024-01-11
- cut the 1.37 branchWed 2024-01-24
- release 1.37.0-rc.1 on testnet with voting planned forMon 2024-01-29
Mon 2024-01-29
- release 1.37.0-rc.2 on testnet with voting moved toMon 2024-02-05
Thu 2024-02-01
- release 1.37.0-rc.3 on testnet with voting moved toTue 2024-02-06
Tue 2024-02-06 12:00:00
- Protocol version 64 voting on testnetTue 2024-02-06 14:20:00
- Start of the resharding epoch on testnetWed 2024-02-07 08:00:00
- Adoption of protocol version 64 on testnet and switch to the new shard layoutTue 2024-03-05
- release 1.37.0 on mainnet with votiing planned forMon 2024-03-11 18:00:00
Fri 2024-03-08
- release 1.37.1 on mainnet without changes to voting. Added 2567e705c5d98ea7365fe3fbcaff5cacc5f81d4a for OOM, 915aea7d9e45a951d172d771d35cf1a40d3f2338 for stack overflow, and 77f40faf452db2815f5174585b6a0607c332dc60Planned events:
Mon 2024-03-11 18:00:00
-1.37.0 voting date on mainnetTue 2024-03-12 07:00:00
- start of resharding epoch on mainnetTuesday 2024-03-12 23:00:00
- 1.37.0 protocol upgrade on mainnet and the start of the first epoch with 5 shards