Hyper Upgrade

[jdisanti]

This tracks progress on the upgrade to Hyper 1.0

Many of these changes are prototyped in this branch which can send an E2E request with RusTLS

Implementation Tasks

Upstream Changes

• [ ] Backport capture_connection and poison to hyper-util
• [x] Create crate for 0.14 -> 1.x bridge. This allows us to use hyper-rustls and hyper-nativetls against the RC (or 1.0)

SDK changes

• [ ] Backwards compatible solution for SegmentedBuf in ByteStream
• [ ] hyper::body::channel() replacement for dvr
• [ ] Careful upgrade + testing of AwsChunkedBody—the Frame API changes the flow here a little bit
• [ ] Update all tests
• [ ] Update Checksum Body
• [ ] Add Spawn to AwsSmithyAsync—bridge this with Hyper Executor (you now must provide an executor to create a Hyper builder)
• [ ] Event Stream Upgrade

Testing Tasks

• [ ] Simple request/response with:
  • [ ] Native TLS
  • [ ] Rustls
• [ ] Streaming request/response with:
  • [ ] Native TLS
  • [ ] Rustls
• [ ] Checksum validating response (S3 with checksums)
• [ ] Checksum calculating request (S3 with checksums)
• [ ] Unidirectional event stream operation (e.g., Transcribe Streaming and S3 Select):
  • [ ] HTTP/1.1
  • [ ] h2
  • [ ] Native TLS
  • [ ] Rustls
• [ ] Bidirectional event stream operation (Pokemon client/server test?):
  • [ ] h2
  • [ ] Native TLS
  • [ ] Rustls

[jdisanti]

I spent some time working on this this week. I was able to get aws-smithy-http compiling with most tests passing, and some of aws-smithy-checksums (ended up stubbing out the checksum calculating body to move forward). My work is available on the hyper-1x-experimentation-dec-7-2022 branch. Findings below.

Further work that is unblocked right now:

• [x] Fix failing tests in aws-smithy-http after upgrade
• [x] Reimplement checksum calculating body in aws-smithy-checksums to support new Body trait

Prerequisites before continuing:

• [x] Need hyper-util to release with a Connect/Connection traits and connection pool
• [x] Need an upgraded TLS library (hyper-tls or hyper-rustls) to actually test the SDK. We can do this work ourselves if the previous prerequisite is completed.

Work that can be done after prerequisites:

• [x] Upgrade or wait for hyper-rustls to support hyper 1.x
• [x] Upgrade or wait for hyper-tls to support hyper 1.x
• [ ] Fix dvr in aws-smithy-client (transition from hyper::body::channel() to http_body_util::StreamBody)
• [x] Continue investigation

[jdisanti]

Note: On the Body impls, it should be safe to assume trailers come after data frames since hyper will error out otherwise.

[jdisanti]

Findings so far:

• Depending on what hyper-util ends up supporting, we may need to fully implement the HTTP versioning RFC to avoid needing to worry about ALPN (or it may be cheaper to do so).

Work in progress test matrix:

• [ ] Simple request/response with:
  • [ ] Native TLS
  • [ ] Rustls
• [ ] Streaming request/response with:
  • [ ] Native TLS
  • [ ] Rustls
• [ ] Checksum validating response (S3 with checksums)
• [ ] Checksum calculating request (S3 with checksums)
• [ ] Unidirectional event stream operation (e.g., Transcribe Streaming and S3 Select):
  • [ ] HTTP/1.1
  • [ ] h2
  • [ ] Native TLS
  • [ ] Rustls
• [ ] Bidirectional event stream operation (Pokemon client/server test?):
  • [ ] h2
  • [ ] Native TLS
  • [ ] Rustls