upbqdn commented 3 weeks ago

name: 'Release Checklist Template' about: 'Checklist to create and publish a Zebra release' title: 'Release Zebra (version)' labels: 'A-release, C-trivial, P-Critical :ambulance:' assignees: ''

Prepare for the Release

[x] Make sure there has been at least one successful full sync test since the last state change, or start a manual full sync.
[x] Make sure the PRs with the new checkpoint hashes and missed dependencies are already merged. (See the release ticket checklist for details)

Summarise Release Changes

These steps can be done a few days before the release, in the same PR:

Change Log

Important: Any merge into main deletes any edits to the draft changelog. Once you are ready to tag a release, copy the draft changelog into CHANGELOG.md.

We use the Release Drafter workflow to automatically create a draft changelog. We follow the Keep a Changelog format.

To create the final change log:

[x] Copy the latest draft changelog into CHANGELOG.md (there can be multiple draft releases)
[x] Delete any trivial changes
- [x] Put the list of deleted changelog entries in a PR comment to make reviewing easier
[x] Combine duplicate changes
[x] Edit change descriptions so they will make sense to Zebra users
[x] Check the category for each change
- Prefer the "Fix" category if you're not sure

README

README updates can be skipped for urgent releases.

Update the README to:

[x] Remove any "Known Issues" that have been fixed since the last release.
[x] Update the "Build and Run Instructions" with any new dependencies. Check for changes in the Dockerfile since the last tag: git diff <previous-release-tag> docker/Dockerfile.
[x] If Zebra has started using newer Rust language features or standard library APIs, update the known working Rust version in the README, book, and Cargo.tomls

You can use a command like:

fastmod --fixed-strings '1.58' '1.65'

Create the Release PR

[x] Push the updated changelog and README into a new branch for example: bump-v1.0.0 - this needs to be different to the tag name
[x] Create a release PR by adding &template=release-checklist.md to the comparing url (Example).
[x] Freeze the batched queue using Mergify.
[x] Mark all the release PRs as Critical priority, so they go in the urgent Mergify queue.
[ ] Mark all non-release PRs with do-not-merge, because Mergify checks approved PRs against every commit, even when a queue is frozen.

Update Versions and End of Support

Update Zebra Version

Choose a Release Level

Zebra follows semantic versioning. Semantic versions look like: MAJOR.MINOR.PATCH[-TAG.PRE-RELEASE]

Choose a release level for zebrad. Release levels are based on user-visible changes from the changelog:

Mainnet Network Upgrades are major releases
significant new features or behaviour changes; changes to RPCs, command-line, or configs; and deprecations or removals are minor releases
otherwise, it is a patch release

Zebra's Rust API doesn't have any support or stability guarantees, so we keep all the zebra-* and tower-* crates on a beta pre-release version.

Update Crate Versions

If you're publishing crates for the first time, log in to crates.io, and make sure you're a member of owners group.

Check that the release will work:

[x] Update crate versions, commit the changes to the release branch, and do a release dry-run:

# Update everything except for alpha crates and zebrad:
cargo release version --verbose --execute --allow-branch '*' --workspace --exclude zebrad --exclude zebra-scan --exclude zebra-grpc beta
# Due to a bug in cargo-release, we need to pass exact versions for alpha crates:
cargo release version --verbose --execute --allow-branch '*' --package zebra-scan 0.1.0-alpha.4
cargo release version --verbose --execute --allow-branch '*' --package zebra-grpc 0.1.0-alpha.2
# Update zebrad:
cargo release version --verbose --execute --allow-branch '*' --package zebrad patch # [ major | minor | patch ]
# Continue with the release process:
cargo release replace --verbose --execute --allow-branch '*' --package zebrad
cargo release commit --verbose --execute --allow-branch '*'

Crate publishing is automatically checked in CI using "dry run" mode, however due to a bug in cargo-release we need to pass exact versions to the alpha crates:

[x] Update zebra-scan and zebra-grpc alpha crates in the release-crates-dry-run workflow script
[x] Push the above version changes to the release branch.

Update End of Support

The end of support height is calculated from the current blockchain height:

[x] Find where the Zcash blockchain tip is now by using a Zcash explorer or other tool.
[x] Replace ESTIMATED_RELEASE_HEIGHT in end_of_support.rs with the height you estimate the release will be tagged.

Optional: calculate the release tagging height

- Add `1152` blocks for each day until the release - For example, if the release is in 3 days, add `1152 * 3` to the current Mainnet block height

Update the Release PR

[x] Push the version increments and the release constants to the release branch.

Publish the Zebra Release

Create the GitHub Pre-Release

[x] Wait for all the release PRs to be merged
[x] Create a new release using the draft release as a base, by clicking the Edit icon in the draft release
[x] Set the tag name to the version tag, for example: v1.0.0
[x] Set the release to target the main branch
[x] Set the release title to Zebra followed by the version tag, for example: Zebra 1.0.0
[x] Replace the prepopulated draft changelog in the release description with the final changelog you created; starting just after the title ## [Zebra ... of the current version being released, and ending just before the title of the previous release.
[x] Mark the release as 'pre-release', until it has been built and tested
[x] Publish the pre-release to GitHub using "Publish Release"
[x] Delete all the draft releases from the list of releases

Test the Pre-Release

[x] Wait until the Docker binaries have been built on main, and the quick tests have passed:
- [x] ci-unit-tests-docker.yml
- [x] ci-integration-tests-gcp.yml
[x] Wait until the pre-release deployment machines have successfully launched

Publish Release

[x] Publish the release to GitHub by disabling 'pre-release', then clicking "Set as the latest release"

Publish Crates

[x] Run cargo login
[x] Run cargo clean in the zebra repo (optional)
[x] Publish the crates to crates.io: cargo release publish --verbose --workspace --execute
[x] Check that Zebra can be installed from crates.io: cargo install --locked --force --version 1.minor.patch zebrad && ~/.cargo/bin/zebrad and put the output in a comment on the PR.

Publish Docker Images

[x] Wait for the the Docker images to be published successfully.
[x] Un-freeze the batched queue using Mergify.
[x] Remove do-not-merge from the PRs you added it to

Release Failures

If building or running fails after tagging:

Tag a new release, following these instructions...

1. Fix the bug that caused the failure 2. Start a new `patch` release 3. Skip the **Release Preparation**, and start at the **Release Changes** step 4. Update `CHANGELOG.md` with details about the fix 5. Follow the release checklist for the new Zebra version

upbqdn commented 3 weeks ago

Output from cargo update:

    Updating crates.io index
     Locking 9 packages to latest compatible versions
    Updating hyper-util v0.1.9 -> v0.1.10
    Updating insta v1.40.0 -> v1.41.0
    Updating libm v0.2.8 -> v0.2.11
    Updating quinn-udp v0.5.5 -> v0.5.6
    Updating reqwest v0.12.8 -> v0.12.9
    Updating rustix v0.38.37 -> v0.38.38
    Updating rustls v0.23.15 -> v0.23.16
    Updating serde v1.0.213 -> v1.0.214
    Updating serde_derive v1.0.213 -> v1.0.214
note: pass `--verbose` to see 125 unchanged dependencies behind latest

github-actions[bot] commented 3 weeks ago

:mag: Vulnerabilities of `us-docker.pkg.dev/zfnd-dev-zebra/zebra/zebrad:pr-8979`

:package: Image Reference us-docker.pkg.dev/zfnd-dev-zebra/zebra/zebrad:pr-8979

digest	`sha256:fcc690ac8031c2f8792b2e1b2ce220a7f27d8108fb43be1fc6405e62eb747a86`
vulnerabilities
size	106 MB
packages	114

:package: Base Image debian:12-slim

also known as	`12.7-slim` `bookworm-20241016-slim` `bookworm-slim`
digest	`sha256:d83056144b2dd301730d2739635c8cbdeaaae20d6887146434184f8c060f03ce`
vulnerabilities

stdlib 1.19.8 (golang)

pkg:golang/stdlib@1.19.8
```dockerfile # Dockerfile (219:219) COPY --from=release /usr/local/bin/zebrad /usr/local/bin ```

Affected range	`<1.21.11`
Fixed version	`1.21.11`
EPSS Score	`0.06%`
EPSS Percentile	`28th percentile`

Description

The various Is methods (IsPrivate, IsLoopback, etc) did not work as expected for IPv4-mapped IPv6 addresses, returning false for addresses which would return true in their traditional IPv4 forms.

Affected range	`<1.19.9`
Fixed version	`1.19.9`
EPSS Score	`0.28%`
EPSS Percentile	`69th percentile`

Description

Not all valid JavaScript whitespace characters are considered to be whitespace. Templates containing whitespace characters outside of the character set "\t\n\f\r\u0020\u2028\u2029" in JavaScript contexts that also contain actions may not be properly sanitized during execution.

Affected range	`<1.19.10`
Fixed version	`1.19.10`
EPSS Score	`0.08%`
EPSS Percentile	`35th percentile`

Description

On Unix platforms, the Go runtime does not behave differently when a binary is run with the setuid/setgid bits. This can be dangerous in certain cases, such as when dumping memory state, or assuming the status of standard i/o file descriptors. If a setuid/setgid binary is executed with standard I/O file descriptors closed, opening any files can result in unexpected content being read or written with elevated privileges. Similarly, if a setuid/setgid program is terminated, either via panic or signal, it may leak the contents of its registers.

Affected range	`<1.22.7`
Fixed version	`1.22.7`
EPSS Score	`0.04%`
EPSS Percentile	`17th percentile`

Description

Calling Parse on a "// +build" build tag line with deeply nested expressions can cause a panic due to stack exhaustion.

Affected range	`<1.22.7`
Fixed version	`1.22.7`
EPSS Score	`0.04%`
EPSS Percentile	`17th percentile`

Description

Calling Decoder.Decode on a message which contains deeply nested structures can cause a panic due to stack exhaustion. This is a follow-up to CVE-2022-30635.

Affected range	`<1.21.12`
Fixed version	`1.21.12`
EPSS Score	`0.04%`
EPSS Percentile	`17th percentile`

Description

The net/http HTTP/1.1 client mishandled the case where a server responds to a request with an "Expect: 100-continue" header with a non-informational (200 or higher) status. This mishandling could leave a client connection in an invalid state, where the next request sent on the connection will fail. An attacker sending a request to a net/http/httputil.ReverseProxy proxy can exploit this mishandling to cause a denial of service by sending "Expect: 100-continue" requests which elicit a non-informational response from the backend. Each such request leaves the proxy with an invalid connection, and causes one subsequent request using that connection to fail.

Affected range	`<1.21.8`
Fixed version	`1.21.8`
EPSS Score	`0.04%`
EPSS Percentile	`11th percentile`

Description

The ParseAddressList function incorrectly handles comments (text within parentheses) within display names. Since this is a misalignment with conforming address parsers, it can result in different trust decisions being made by programs using different parsers.

Affected range	`<1.21.9`
Fixed version	`1.21.9`
EPSS Score	`0.04%`
EPSS Percentile	`14th percentile`

Description

An attacker may cause an HTTP/2 endpoint to read arbitrary amounts of header data by sending an excessive number of CONTINUATION frames. Maintaining HPACK state requires parsing and processing all HEADERS and CONTINUATION frames on a connection. When a request's headers exceed MaxHeaderBytes, no memory is allocated to store the excess headers, but they are still parsed. This permits an attacker to cause an HTTP/2 endpoint to read arbitrary amounts of header data, all associated with a request which is going to be rejected. These headers can include Huffman-encoded data which is significantly more expensive for the receiver to decode than for an attacker to send. The fix sets a limit on the amount of excess header frames we will process before closing a connection.

Affected range	`<1.20.0`
Fixed version	`1.20.0`
EPSS Score	`0.07%`
EPSS Percentile	`32nd percentile`

Description

Before Go 1.20, the RSA based TLS key exchanges used the math/big library, which is not constant time. RSA blinding was applied to prevent timing attacks, but analysis shows this may not have been fully effective. In particular it appears as if the removal of PKCS#1 padding may leak timing information, which in turn could be used to recover session key bits. In Go 1.20, the crypto/tls library switched to a fully constant time RSA implementation, which we do not believe exhibits any timing side channels.

Affected range	`<1.20.11`
Fixed version	`1.20.11`
EPSS Score	`0.11%`
EPSS Percentile	`46th percentile`

Description

The filepath package does not recognize paths with a \??\ prefix as special. On Windows, a path beginning with \??\ is a Root Local Device path equivalent to a path beginning with \\?\. Paths with a \??\ prefix may be used to access arbitrary locations on the system. For example, the path \??\c:\x is equivalent to the more common path c:\x. Before fix, Clean could convert a rooted path such as \a\..\??\b into the root local device path \??\b. Clean will now convert this to .\??\b. Similarly, Join(\, ??, b) could convert a seemingly innocent sequence of path elements into the root local device path \??\b. Join will now convert this to \.\??\b. In addition, with fix, IsAbs now correctly reports paths beginning with \??\ as absolute, and VolumeName correctly reports the \??\ prefix as a volume name. UPDATE: Go 1.20.11 and Go 1.21.4 inadvertently changed the definition of the volume name in Windows paths starting with \?, resulting in filepath.Clean(\?\c:) returning \?\c: rather than \?\c:\ (among other effects). The previous behavior has been restored.

Affected range	`<1.20.10`
Fixed version	`1.20.10`
EPSS Score	`83.78%`
EPSS Percentile	`99th percentile`

Description

A malicious HTTP/2 client which rapidly creates requests and immediately resets them can cause excessive server resource consumption. While the total number of requests is bounded by the http2.Server.MaxConcurrentStreams setting, resetting an in-progress request allows the attacker to create a new request while the existing one is still executing. With the fix applied, HTTP/2 servers now bound the number of simultaneously executing handler goroutines to the stream concurrency limit (MaxConcurrentStreams). New requests arriving when at the limit (which can only happen after the client has reset an existing, in-flight request) will be queued until a handler exits. If the request queue grows too large, the server will terminate the connection. This issue is also fixed in golang.org/x/net/http2 for users manually configuring HTTP/2. The default stream concurrency limit is 250 streams (requests) per HTTP/2 connection. This value may be adjusted using the golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams setting and the ConfigureServer function.

Affected range	`<1.20.10`
Fixed version	`1.20.10`
EPSS Score	`0.42%`
EPSS Percentile	`75th percentile`

Description

A malicious HTTP/2 client which rapidly creates requests and immediately resets them can cause excessive server resource consumption. While the total number of requests is bounded by the http2.Server.MaxConcurrentStreams setting, resetting an in-progress request allows the attacker to create a new request while the existing one is still executing. With the fix applied, HTTP/2 servers now bound the number of simultaneously executing handler goroutines to the stream concurrency limit (MaxConcurrentStreams). New requests arriving when at the limit (which can only happen after the client has reset an existing, in-flight request) will be queued until a handler exits. If the request queue grows too large, the server will terminate the connection. This issue is also fixed in golang.org/x/net/http2 for users manually configuring HTTP/2. The default stream concurrency limit is 250 streams (requests) per HTTP/2 connection. This value may be adjusted using the golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams setting and the ConfigureServer function.

Affected range	`<1.22.7`
Fixed version	`1.22.7`
EPSS Score	`0.19%`
EPSS Percentile	`56th percentile`

Description

Calling Decoder.Decode on a message which contains deeply nested structures can cause a panic due to stack exhaustion. This is a follow-up to CVE-2022-30635.

Affected range	`<1.19.9`
Fixed version	`1.19.9`
EPSS Score	`0.14%`
EPSS Percentile	`50th percentile`

Description

Templates containing actions in unquoted HTML attributes (e.g. "attr={{.}}") executed with empty input can result in output with unexpected results when parsed due to HTML normalization rules. This may allow injection of arbitrary attributes into tags.

Affected range	`<1.19.9`
Fixed version	`1.19.9`
EPSS Score	`0.14%`
EPSS Percentile	`50th percentile`

Description

Angle brackets (<>) are not considered dangerous characters when inserted into CSS contexts. Templates containing multiple actions separated by a '/' character can result in unexpectedly closing the CSS context and allowing for injection of unexpected HTML, if executed with untrusted input.

github-actions[bot] commented 3 weeks ago

Recommended fixes for image `us-docker.pkg.dev/zfnd-dev-zebra/zebra/zebrad:pr-8979`

Base image is debian:bookworm-slim

Name	bookworm-20241016-slim
Digest	sha256:d83056144b2dd301730d2739635c8cbdeaaae20d6887146434184f8c060f03ce
Vulnerabilities
Pushed	1 week ago
Size	29 MB
Packages	125
Flavor	debian
OS	12
Slim	✅

The base image is also available under the supported tag(s): 12-slim, 12.7-slim, bookworm-20241016-slim

Refresh base image

Rebuild the image using a newer base image version. Updating this may result in breaking changes.

✅ This image version is up to date.

Change base image

Tag	Details	Pushed
`stable-slim` Tag is preferred tag Also known as: stable-20241016-slim	Benefits: Same OS detected Tag is preferred tag Tag was pushed more recently Image has similar size Image has same number of vulnerabilities Image contains equal number of packages Tag is using slim variant stable-slim is the fourth most popular tag with 46K pulls per month Image details: Size: 29 MB Flavor: debian OS: 12 Slim: ✅	1 week ago
`stable` Image has same number of vulnerabilities Also known as: stable-20241016	Benefits: Same OS detected Tag was pushed more recently Image has same number of vulnerabilities Image contains equal number of packages stable is the 7th most popular tag with 32K pulls per month Image details: Size: 50 MB Flavor: debian OS: 12	1 week ago
`bookworm` Tag is latest Also known as: 12.7 12 bookworm-20241016 latest	Benefits: Same OS detected Tag is latest Image has same number of vulnerabilities Image contains equal number of packages bookworm is the 10th most popular tag with 14K pulls per month Image details: Size: 50 MB Flavor: debian OS: 12	1 week ago
`testing-slim` Major OS version update Also known as: testing-20241016-slim	Benefits: Same OS detected Tag was pushed more recently Image has similar size Major OS version update Image contains similar number of packages Tag is using slim variant testing-slim is the sixth most popular tag with 33K pulls per month Image details: Size: 32 MB Flavor: debian OS: 13 Slim: ✅	1 week ago
`sid-slim` Major OS version update Also known as: sid-20241016-slim	Benefits: Same OS detected Tag was pushed more recently Image has similar size Major OS version update Image contains similar number of packages Tag is using slim variant sid-slim is the 9th most popular tag with 15K pulls per month Image details: Size: 32 MB Flavor: debian OS: 13 Slim: ✅	1 week ago

github-actions[bot] commented 3 weeks ago

Overview

Image reference	`zfnd/zebra:latest`	`us-docker.pkg.dev/zfnd-dev-zebra/zebra/zebrad:pr-8979`
- digest	`64630802fdcb`	`fcc690ac8031`
- tag	`latest`	`pr-8979`
- provenance	https://github.com/ZcashFoundation/zebra/commit/f45f6f282c7b1b3e82594de328a2d4f1849bc99a	https://github.com/ZcashFoundation/zebra/commit/4c5b5590b811ef84ae8985d6334387b44690f06d
- vulnerabilities
- platform	linux/amd64	linux/amd64
- size	106 MB	106 MB (+8.4 kB)
- packages	114	114
Base Image	`debian:bookworm-slim` also known as: • `12-slim` • `12.7-slim` • `bookworm-20241016-slim`	`debian:bookworm-slim` also known as: • `12-slim` • `12.7-slim` • `bookworm-20241016-slim`
- vulnerabilities

Labels (3 changes)

> * `±` 3 changed > * _5 unchanged_ ```diff -org.opencontainers.image.created=2024-10-25T23:14:55.621Z +org.opencontainers.image.created=2024-10-30T08:57:33.145Z org.opencontainers.image.description=Zcash - Financial Privacy in Rust 🦓 org.opencontainers.image.licenses=Apache-2.0 -org.opencontainers.image.revision=f45f6f282c7b1b3e82594de328a2d4f1849bc99a +org.opencontainers.image.revision=4c5b5590b811ef84ae8985d6334387b44690f06d org.opencontainers.image.source=https://github.com/ZcashFoundation/zebra org.opencontainers.image.title=zebra org.opencontainers.image.url=https://github.com/ZcashFoundation/zebra -org.opencontainers.image.version=2.0.0 +org.opencontainers.image.version=pr-8979 ```

arya2 commented 3 weeks ago

The test_grpc_response_data test seemed to hang here: https://github.com/ZcashFoundation/zebra/actions/runs/11589846881/job/32266208300?pr=8979#step:12:2012

Re-running it, I expect it'll pass now since the other unit test jobs did.

ZcashFoundation / zebra

chore: Release v2.0.1 #8979

Prepare for the Release

Summarise Release Changes

Change Log

README

Create the Release PR

Update Versions and End of Support

Update Zebra Version

Choose a Release Level

Update Crate Versions

Update End of Support

Update the Release PR

Publish the Zebra Release

Create the GitHub Pre-Release

Test the Pre-Release

Publish Release

Publish Crates

Publish Docker Images

Release Failures

:mag: Vulnerabilities of `us-docker.pkg.dev/zfnd-dev-zebra/zebra/zebrad:pr-8979`

Recommended fixes for image `us-docker.pkg.dev/zfnd-dev-zebra/zebra/zebrad:pr-8979`

Refresh base image

Change base image

Overview

ZcashFoundation / zebra

chore: Release v2.0.1 #8979

Prepare for the Release

Summarise Release Changes

Change Log

README

Create the Release PR

Update Versions and End of Support

Update Zebra Version

Choose a Release Level

Update Crate Versions

Update End of Support

Update the Release PR

Publish the Zebra Release

Create the GitHub Pre-Release

Test the Pre-Release

Publish Release

Publish Crates

Publish Docker Images

Release Failures

:mag: Vulnerabilities of us-docker.pkg.dev/zfnd-dev-zebra/zebra/zebrad:pr-8979

Recommended fixes for image us-docker.pkg.dev/zfnd-dev-zebra/zebra/zebrad:pr-8979

Refresh base image

Change base image

Overview

:mag: Vulnerabilities of `us-docker.pkg.dev/zfnd-dev-zebra/zebra/zebrad:pr-8979`

Recommended fixes for image `us-docker.pkg.dev/zfnd-dev-zebra/zebra/zebrad:pr-8979`