Adding new gradle tasks for generating Cargo.lock

landonxjames commented 2 weeks ago

Motivation and Context

We have had numerous recent issues where updated dependencies have broken our build/release process. This seeks to fix that in a brute force way by manually generating Cargo.lock files for our main workspaces.

Description

Introduce a new gradle task generateAllLockfiles (and some child tasks that it invokes) that generates lock files for

aws-config crate
aws/rust-runtime workspace
rust-runtime workspace
the superset workspace generated by the assemble task
- NOTE: This Cargo.lock is actually generated from the workspace in the aws-sdk-rust repo, saved in this repo, and copied into place at build time. Generating this one requires a local copy of the aws-sdk-rust repo whose absolute path is indicated by the -Paws-sdk-rust-path parameter.

Testing

I ran our E2E tests against 3 commits to confirm that the Cargo.lock files operate as we expect:

9d84980ee2e37deadb7e45450d65c5e6bc69e07c The head commit from this PR which has our Cargo.lock files
- ✅ Passes
d9d9fcec2de0e898bcc804ee0bb31393d7098e72 with lockfiles and with the httparse and crc323c crates unpinned
- ✅ Passes
df75beb160890b2a9c382c8a2f7da8d7d0ddd3c8 with no lockfiles and has the httparse and crc32c crates unpinned
- ❌ Fails (as expected)

This series of tests confirms that that Cargo.lock files will protect us against bad dependency updates going forward.

Checklist

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

github-actions[bot] commented 2 weeks ago

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions[bot] commented 2 weeks ago

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions[bot] commented 2 weeks ago

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions[bot] commented 2 weeks ago

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions[bot] commented 2 weeks ago

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions[bot] commented 2 weeks ago

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions[bot] commented 2 weeks ago

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions[bot] commented 2 weeks ago

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions[bot] commented 2 weeks ago

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions[bot] commented 2 weeks ago

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions[bot] commented 2 weeks ago

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

landonxjames commented 2 weeks ago

For 3, we probably need to place aws/sdk/Cargo.lock into a shared test workspace (can't remember off the top of my head which Kotlin code manages the location). If we revert https://github.com/smithy-lang/smithy-rs/commit/0cbeef351b1021e4d55a91a5e1c78f8d08e2012d and comment out logger.fine(("cargo update").runCommand(testDir)), let smithy-rs CI run, then a CI task check-sdk-codegen-unit-tests may fail even with changes in this PR (if so, we do need to handle 3).

Gave that a shot in this PR: https://github.com/smithy-lang/smithy-rs/pull/3700. With the version pinning and the cargo update removed. The CI succeeded, but that doesn't necessarily mean that this PR fixed it as it was (it likely doesn't since the tests generate their own shared workspace).

Added https://github.com/smithy-lang/smithy-rs/pull/3689/commits/48eb3adf3ee57bb8da498b69653ee21bb740ae41 to copy the (superset) Cargo.lock file into the generated test workspaces

github-actions[bot] commented 2 weeks ago

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

smithy-lang / smithy-rs