aws-config feature flag "client-hyper" ships native-tls and rustls dependencies

lcmgh commented 1 year ago

Describe the bug

Hi!

I am in the process of switching from openssl to rustls so I scanned my project for native-tls. It looks like aws-config ships rustls and native-tls if I am not mistaken. I would like to include rustls only when the appropriate feature flags are set to ensure my app will work on images without OpenSSL.

I am also curious how it is determined which tls lib the hyper client should use?

Cargo.toml

[dependencies]
aws-config = { version = "0.53.0", default-features = false, features = ["client-hyper"] }

..

[features]
native-tls = ["aws-sdk-kinesis/native-tls", "aws-sdk-dynamodb/native-tls", "aws-smithy-client/native-tls", "aws-config/native-tls", "hyper-proxy/tls", "hyper-proxy/native-tls"]
rustls = ["aws-sdk-kinesis/rustls", "aws-sdk-dynamodb/rustls", "aws-smithy-client/rustls", "aws-config/rustls", "hyper-proxy/rustls"]

default = ["native-tls"]

cargo tree --features rustls --no-default-features >> tree.txt

├── aws-config v0.53.0
│   ├── aws-credential-types v0.53.0
│   │   ├── aws-smithy-async v0.53.1
│   │   │   ├── futures-util v0.3.25
│   │   │   │   ├── futures-channel v0.3.25
│   │   │   │   │   ├── futures-core v0.3.25
│   │   │   │   │   └── futures-sink v0.3.25
│   │   │   │   ├── futures-core v0.3.25
│   │   │   │   ├── futures-io v0.3.25
│   │   │   │   ├── futures-macro v0.3.25 (proc-macro)
│   │   │   │   │   ├── proc-macro2 v1.0.50 (*)
│   │   │   │   │   ├── quote v1.0.23 (*)
│   │   │   │   │   └── syn v1.0.107 (*)
│   │   │   │   ├── futures-sink v0.3.25
│   │   │   │   ├── futures-task v0.3.25
│   │   │   │   ├── memchr v2.5.0
│   │   │   │   ├── pin-project-lite v0.2.9
│   │   │   │   ├── pin-utils v0.1.0
│   │   │   │   └── slab v0.4.7
│   │   │   │       [build-dependencies]
│   │   │   │       └── autocfg v1.1.0
│   │   │   ├── pin-project-lite v0.2.9
│   │   │   ├── tokio v1.24.2
│   │   │   │   ├── bytes v1.3.0
│   │   │   │   ├── libc v0.2.139
│   │   │   │   ├── memchr v2.5.0
│   │   │   │   ├── mio v0.8.5
│   │   │   │   │   ├── libc v0.2.139
│   │   │   │   │   └── log v0.4.17
│   │   │   │   │       └── cfg-if v1.0.0
│   │   │   │   ├── num_cpus v1.15.0
│   │   │   │   │   └── libc v0.2.139
│   │   │   │   ├── parking_lot v0.12.1
│   │   │   │   │   ├── lock_api v0.4.9
│   │   │   │   │   │   └── scopeguard v1.1.0
│   │   │   │   │   │   [build-dependencies]
│   │   │   │   │   │   └── autocfg v1.1.0
│   │   │   │   │   └── parking_lot_core v0.9.6
│   │   │   │   │       ├── cfg-if v1.0.0
│   │   │   │   │       ├── libc v0.2.139
│   │   │   │   │       └── smallvec v1.10.0
│   │   │   │   ├── pin-project-lite v0.2.9
│   │   │   │   ├── signal-hook-registry v1.4.0
│   │   │   │   │   └── libc v0.2.139
│   │   │   │   ├── socket2 v0.4.7
│   │   │   │   │   └── libc v0.2.139
│   │   │   │   └── tokio-macros v1.8.2 (proc-macro)
│   │   │   │       ├── proc-macro2 v1.0.50 (*)
│   │   │   │       ├── quote v1.0.23 (*)
│   │   │   │       └── syn v1.0.107 (*)
│   │   │   │   [build-dependencies]
│   │   │   │   └── autocfg v1.1.0
│   │   │   └── tokio-stream v0.1.11
│   │   │       ├── futures-core v0.3.25
│   │   │       ├── pin-project-lite v0.2.9
│   │   │       └── tokio v1.24.2 (*)
│   │   ├── aws-smithy-types v0.53.1
│   │   │   ├── base64-simd v0.7.0
│   │   │   │   └── simd-abstraction v0.7.1
│   │   │   │       └── outref v0.1.0
│   │   │   ├── itoa v1.0.5
│   │   │   ├── num-integer v0.1.45
│   │   │   │   └── num-traits v0.2.15
│   │   │   │       [build-dependencies]
│   │   │   │       └── autocfg v1.1.0
│   │   │   │   [build-dependencies]
│   │   │   │   └── autocfg v1.1.0
│   │   │   ├── ryu v1.0.12
│   │   │   └── time v0.3.17
│   │   │       └── time-core v0.1.0
│   │   ├── tokio v1.24.2 (*)
│   │   ├── tracing v0.1.37
│   │   │   ├── cfg-if v1.0.0
│   │   │   ├── log v0.4.17 (*)
│   │   │   ├── pin-project-lite v0.2.9
│   │   │   ├── tracing-attributes v0.1.23 (proc-macro)
│   │   │   │   ├── proc-macro2 v1.0.50 (*)
│   │   │   │   ├── quote v1.0.23 (*)
│   │   │   │   └── syn v1.0.107 (*)
│   │   │   └── tracing-core v0.1.30
│   │   │       └── once_cell v1.17.0
│   │   └── zeroize v1.5.7
│   ├── aws-http v0.53.0
│   │   ├── aws-credential-types v0.53.0 (*)
│   │   ├── aws-smithy-http v0.53.1
│   │   │   ├── aws-smithy-types v0.53.1 (*)
│   │   │   ├── bytes v1.3.0
│   │   │   ├── bytes-utils v0.1.3
│   │   │   │   ├── bytes v1.3.0
│   │   │   │   └── either v1.8.0
│   │   │   ├── futures-core v0.3.25
│   │   │   ├── http v0.2.8
│   │   │   │   ├── bytes v1.3.0
│   │   │   │   ├── fnv v1.0.7
│   │   │   │   └── itoa v1.0.5
│   │   │   ├── http-body v0.4.5
│   │   │   │   ├── bytes v1.3.0
│   │   │   │   ├── http v0.2.8 (*)
│   │   │   │   └── pin-project-lite v0.2.9
│   │   │   ├── hyper v0.14.23
│   │   │   │   ├── bytes v1.3.0
│   │   │   │   ├── futures-channel v0.3.25 (*)
│   │   │   │   ├── futures-core v0.3.25
│   │   │   │   ├── futures-util v0.3.25 (*)
│   │   │   │   ├── h2 v0.3.15
│   │   │   │   │   ├── bytes v1.3.0
│   │   │   │   │   ├── fnv v1.0.7
│   │   │   │   │   ├── futures-core v0.3.25
│   │   │   │   │   ├── futures-sink v0.3.25
│   │   │   │   │   ├── futures-util v0.3.25 (*)
│   │   │   │   │   ├── http v0.2.8 (*)
│   │   │   │   │   ├── indexmap v1.9.2
│   │   │   │   │   │   └── hashbrown v0.12.3
│   │   │   │   │   │   [build-dependencies]
│   │   │   │   │   │   └── autocfg v1.1.0
│   │   │   │   │   ├── slab v0.4.7 (*)
│   │   │   │   │   ├── tokio v1.24.2 (*)
│   │   │   │   │   ├── tokio-util v0.7.4
│   │   │   │   │   │   ├── bytes v1.3.0
│   │   │   │   │   │   ├── futures-core v0.3.25
│   │   │   │   │   │   ├── futures-sink v0.3.25
│   │   │   │   │   │   ├── pin-project-lite v0.2.9
│   │   │   │   │   │   ├── tokio v1.24.2 (*)
│   │   │   │   │   │   └── tracing v0.1.37 (*)
│   │   │   │   │   └── tracing v0.1.37 (*)
│   │   │   │   ├── http v0.2.8 (*)
│   │   │   │   ├── http-body v0.4.5 (*)
│   │   │   │   ├── httparse v1.8.0
│   │   │   │   ├── httpdate v1.0.2
│   │   │   │   ├── itoa v1.0.5
│   │   │   │   ├── pin-project-lite v0.2.9
│   │   │   │   ├── socket2 v0.4.7 (*)
│   │   │   │   ├── tokio v1.24.2 (*)
│   │   │   │   ├── tower-service v0.3.2
│   │   │   │   ├── tracing v0.1.37 (*)
│   │   │   │   └── want v0.3.0
│   │   │   │       ├── log v0.4.17 (*)
│   │   │   │       └── try-lock v0.2.4
│   │   │   ├── once_cell v1.17.0
│   │   │   ├── percent-encoding v2.2.0
│   │   │   ├── pin-project-lite v0.2.9
│   │   │   ├── pin-utils v0.1.0
│   │   │   └── tracing v0.1.37 (*)
│   │   ├── aws-smithy-types v0.53.1 (*)
│   │   ├── aws-types v0.53.0
│   │   │   ├── aws-credential-types v0.53.0 (*)
│   │   │   ├── aws-smithy-async v0.53.1 (*)
│   │   │   ├── aws-smithy-client v0.53.1
│   │   │   │   ├── aws-smithy-async v0.53.1 (*)
│   │   │   │   ├── aws-smithy-http v0.53.1 (*)
│   │   │   │   ├── aws-smithy-http-tower v0.53.1
│   │   │   │   │   ├── aws-smithy-http v0.53.1 (*)
│   │   │   │   │   ├── aws-smithy-types v0.53.1 (*)
│   │   │   │   │   ├── bytes v1.3.0
│   │   │   │   │   ├── http v0.2.8 (*)
│   │   │   │   │   ├── http-body v0.4.5 (*)
│   │   │   │   │   ├── pin-project-lite v0.2.9
│   │   │   │   │   ├── tower v0.4.13
│   │   │   │   │   │   ├── futures-core v0.3.25
│   │   │   │   │   │   ├── futures-util v0.3.25 (*)
│   │   │   │   │   │   ├── pin-project v1.0.12
│   │   │   │   │   │   │   └── pin-project-internal v1.0.12 (proc-macro)
│   │   │   │   │   │   │       ├── proc-macro2 v1.0.50 (*)
│   │   │   │   │   │   │       ├── quote v1.0.23 (*)
│   │   │   │   │   │   │       └── syn v1.0.107 (*)
│   │   │   │   │   │   ├── pin-project-lite v0.2.9
│   │   │   │   │   │   ├── tokio v1.24.2 (*)
│   │   │   │   │   │   ├── tower-layer v0.3.2
│   │   │   │   │   │   ├── tower-service v0.3.2
│   │   │   │   │   │   └── tracing v0.1.37 (*)
│   │   │   │   │   └── tracing v0.1.37 (*)
│   │   │   │   ├── aws-smithy-types v0.53.1 (*)
│   │   │   │   ├── bytes v1.3.0
│   │   │   │   ├── fastrand v1.8.0
│   │   │   │   ├── http v0.2.8 (*)
│   │   │   │   ├── http-body v0.4.5 (*)
│   │   │   │   ├── hyper v0.14.23 (*)
│   │   │   │   ├── hyper-rustls v0.23.2
│   │   │   │   │   ├── http v0.2.8 (*)
│   │   │   │   │   ├── hyper v0.14.23 (*)
│   │   │   │   │   ├── log v0.4.17 (*)
│   │   │   │   │   ├── rustls v0.20.8
│   │   │   │   │   │   ├── log v0.4.17 (*)
│   │   │   │   │   │   ├── ring v0.16.20
│   │   │   │   │   │   │   └── untrusted v0.7.1
│   │   │   │   │   │   │   [build-dependencies]
│   │   │   │   │   │   │   └── cc v1.0.78
│   │   │   │   │   │   ├── sct v0.7.0
│   │   │   │   │   │   │   ├── ring v0.16.20 (*)
│   │   │   │   │   │   │   └── untrusted v0.7.1
│   │   │   │   │   │   └── webpki v0.22.0
│   │   │   │   │   │       ├── ring v0.16.20 (*)
│   │   │   │   │   │       └── untrusted v0.7.1
│   │   │   │   │   ├── rustls-native-certs v0.6.2
│   │   │   │   │   │   ├── rustls-pemfile v1.0.2
│   │   │   │   │   │   │   └── base64 v0.21.0
│   │   │   │   │   │   └── security-framework v2.8.1
│   │   │   │   │   │       ├── bitflags v1.3.2
│   │   │   │   │   │       ├── core-foundation v0.9.3
│   │   │   │   │   │       │   ├── core-foundation-sys v0.8.3
│   │   │   │   │   │       │   └── libc v0.2.139
│   │   │   │   │   │       ├── core-foundation-sys v0.8.3
│   │   │   │   │   │       ├── libc v0.2.139
│   │   │   │   │   │       └── security-framework-sys v2.8.0
│   │   │   │   │   │           ├── core-foundation-sys v0.8.3
│   │   │   │   │   │           └── libc v0.2.139
│   │   │   │   │   ├── tokio v1.24.2 (*)
│   │   │   │   │   └── tokio-rustls v0.23.4
│   │   │   │   │       ├── rustls v0.20.8 (*)
│   │   │   │   │       ├── tokio v1.24.2 (*)
│   │   │   │   │       └── webpki v0.22.0 (*)
│   │   │   │   ├── hyper-tls v0.5.0
│   │   │   │   │   ├── bytes v1.3.0
│   │   │   │   │   ├── hyper v0.14.23 (*)
│   │   │   │   │   ├── native-tls v0.2.11

Expected Behavior

Additional feature flag to use hyper rustls or hyper native-tls.

Current Behavior

hyper-client includes dependencies for rustls and native-tls.

Reproduction Steps

See above.

Possible Solution

Add new feature flags

Additional Information/Context

No response

Version

├── aws-arn v0.3.1
├── aws-config v0.53.0
│   ├── aws-credential-types v0.53.0
│   │   ├── aws-smithy-async v0.53.1
│   │   ├── aws-smithy-types v0.53.1
│   ├── aws-http v0.53.0
│   │   ├── aws-credential-types v0.53.0 (*)
│   │   ├── aws-smithy-http v0.53.1
│   │   │   ├── aws-smithy-types v0.53.1 (*)
│   │   ├── aws-smithy-types v0.53.1 (*)
│   │   ├── aws-types v0.53.0
│   │   │   ├── aws-credential-types v0.53.0 (*)
│   │   │   ├── aws-smithy-async v0.53.1 (*)
│   │   │   ├── aws-smithy-client v0.53.1
│   │   │   │   ├── aws-smithy-async v0.53.1 (*)
│   │   │   │   ├── aws-smithy-http v0.53.1 (*)
│   │   │   │   ├── aws-smithy-http-tower v0.53.1
│   │   │   │   │   ├── aws-smithy-http v0.53.1 (*)
│   │   │   │   │   ├── aws-smithy-types v0.53.1 (*)
│   │   │   │   ├── aws-smithy-types v0.53.1 (*)
│   │   │   ├── aws-smithy-http v0.53.1 (*)
│   │   │   ├── aws-smithy-types v0.53.1 (*)
│   ├── aws-sdk-sso v0.23.0
│   │   ├── aws-credential-types v0.53.0 (*)
│   │   ├── aws-endpoint v0.53.0
│   │   │   ├── aws-smithy-http v0.53.1 (*)
│   │   │   ├── aws-smithy-types v0.53.1 (*)
│   │   │   ├── aws-types v0.53.0 (*)
│   │   ├── aws-http v0.53.0 (*)
│   │   ├── aws-sig-auth v0.53.0
│   │   │   ├── aws-credential-types v0.53.0 (*)
│   │   │   ├── aws-sigv4 v0.53.0
│   │   │   │   ├── aws-smithy-http v0.53.1 (*)
│   │   │   ├── aws-smithy-http v0.53.1 (*)
│   │   │   ├── aws-types v0.53.0 (*)
│   │   ├── aws-smithy-async v0.53.1 (*)
│   │   ├── aws-smithy-client v0.53.1 (*)
│   │   ├── aws-smithy-http v0.53.1 (*)
│   │   ├── aws-smithy-http-tower v0.53.1 (*)
│   │   ├── aws-smithy-json v0.53.1
│   │   │   └── aws-smithy-types v0.53.1 (*)
│   │   ├── aws-smithy-types v0.53.1 (*)
│   │   ├── aws-types v0.53.0 (*)
│   ├── aws-sdk-sts v0.23.0
│   │   ├── aws-credential-types v0.53.0 (*)
│   │   ├── aws-endpoint v0.53.0 (*)
│   │   ├── aws-http v0.53.0 (*)
│   │   ├── aws-sig-auth v0.53.0 (*)
│   │   ├── aws-smithy-async v0.53.1 (*)
│   │   ├── aws-smithy-client v0.53.1 (*)
│   │   ├── aws-smithy-http v0.53.1 (*)
│   │   ├── aws-smithy-http-tower v0.53.1 (*)
│   │   ├── aws-smithy-json v0.53.1 (*)
│   │   ├── aws-smithy-query v0.53.1
│   │   │   ├── aws-smithy-types v0.53.1 (*)
│   │   ├── aws-smithy-types v0.53.1 (*)
│   │   ├── aws-smithy-xml v0.53.1
│   │   ├── aws-types v0.53.0 (*)
│   ├── aws-smithy-async v0.53.1 (*)
│   ├── aws-smithy-client v0.53.1 (*)
│   ├── aws-smithy-http v0.53.1 (*)
│   ├── aws-smithy-http-tower v0.53.1 (*)
│   ├── aws-smithy-json v0.53.1 (*)
│   ├── aws-smithy-types v0.53.1 (*)
│   ├── aws-types v0.53.0 (*)
├── aws-sdk-dynamodb v0.23.0
│   ├── aws-credential-types v0.53.0 (*)
│   ├── aws-endpoint v0.53.0 (*)
│   ├── aws-http v0.53.0 (*)
│   ├── aws-sig-auth v0.53.0 (*)
│   ├── aws-smithy-async v0.53.1 (*)
│   ├── aws-smithy-client v0.53.1 (*)
│   ├── aws-smithy-http v0.53.1 (*)
│   ├── aws-smithy-http-tower v0.53.1 (*)
│   ├── aws-smithy-json v0.53.1 (*)
│   ├── aws-smithy-types v0.53.1 (*)
│   ├── aws-types v0.53.0 (*)
├── aws-sdk-kinesis v0.23.0
│   ├── aws-credential-types v0.53.0 (*)
│   ├── aws-endpoint v0.53.0 (*)
│   ├── aws-http v0.53.0 (*)
│   ├── aws-sig-auth v0.53.0 (*)
│   ├── aws-smithy-async v0.53.1 (*)
│   ├── aws-smithy-client v0.53.1 (*)
│   ├── aws-smithy-http v0.53.1 (*)
│   ├── aws-smithy-http-tower v0.53.1 (*)
│   ├── aws-smithy-json v0.53.1 (*)
│   ├── aws-smithy-types v0.53.1 (*)
│   ├── aws-types v0.53.0 (*)
├── aws-smithy-client v0.53.1 (*)
├── aws-types v0.53.0 (*)
│   ├── aws-sdk-dynamodb v0.23.0 (*)
│   │   ├── aws-sdk-dynamodb v0.23.0 (*)

Environment details (OS name and version, etc.)

macOS Montery

Logs

No response

ysaito1001 commented 1 year ago

I am also curious how it is determined which tls lib the hyper client should use?

That's up to how a client crate of aws-smithy-client enables the cargo features native-tls and rustls. When depending on an SDK crate, which is usually the case, the client gets rustls, as shown in the default feature in aws-config (if the client directly depends on aws-smithy-client without enabling native-tls or rustls, no tls library has been selected for hyper client).

To better assist, could you paste your complete Cargo.toml so that we can reproduce what you are observing on our end?

github-actions[bot] commented 1 year ago

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.

lcmgh commented 1 year ago

Hi @ysaito1001 . Here is my Cargo.toml

[package]
name = "myapp"
version = "1.1.2"
edition = "2021"
default-run = "myapp"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
# AWS SDK
aws-config =  { version = "0.53.0", default-features = false, features = ["rustls"] }
aws-smithy-client = { version = "0.54.0", features = ["client-hyper", "rustls"] }
aws-sdk-kinesis = { version = "0.23.0", default-features = false, features = ["rustls"] }
aws-sdk-dynamodb = { version = "0.23.0", default-features = false, features = ["rustls"] }
aws-types = { version = "0.53.0", default-features = false }
tokio-retry = "0.3"
# App settings (read from env, file etc.)
config = { version = "0.13.1", features = ["toml"], default-features = false }

# Serialization
serde = { version = "1.0.138", features = ["derive"] }
serde_json = "1.0.82"

# Kinesis client lib
kinesis-client = { git = "ssh://git@xxx/kinesis-client-rs.git", rev = "v0.1.3", features = ["rustls"] }

# Parallel iterators for utilizing all cpu cores
rayon = "1.5.3"
# Spawn CPU heavy task in rayon tokio threads
tokio-rayon = "2.1.0"

# Async runtime
tokio = { version = "1.19.2", features = ["full"] }

# Logging
tracing = "0.1.35"
tracing-subscriber = { version = "0.3.14", features = ["tracing-log", "json"] }

# Metrics
metrics-exporter-prometheus = { version = "0.11.0", default-features = false, features = ["http-listener"] }
metrics = "0.20.1"

# SQL driver
sqlx = { version = "0.6.0", features = [ "runtime-tokio-rustls", "postgres" ] }

# Derive error types
thiserror = "1.0.31"
# Generic error type for results
eyre = "0.6.8"
testcontainers = { version = "0.14.0", optional = true }
hyper = "0.14.19"

compact_str = { version = "0.6.1", features = ["serde", "bytes"] }

# Gzip
flate2 = "1.0.24"
# To create random mileages in test set / stub
fastrand = { version = "1.7.0", optional = true }

[features]
default = ["local"]
# Stub dependencies
local = ["testcontainers", "fastrand"]

[dev-dependencies]
# Spawns docker containers
testcontainers = "0.14.0"
# Provides callback if variables goes out of scope
scopeguard = "1.1.0"
# http client
reqwest = { version = "0.11.11", features = ["json", "rustls"], default-features = false }
# For some reason mileages are stored as psql NUMERIC in database
rust_decimal = "1.25.0"
# Psql client
sqlx = { version = "0.6.0", features = [ "runtime-tokio-rustls", "postgres", "decimal" ] }
# Time lib for db date fields
# Benchmark lib
criterion = "0.4.0"
tokio-retry = "0.3"

[[bin]]
name = "myapp"

[[bin]]
name = "stub"
path = "src/stub.rs" 
required-features = ["local"]

[[bench]]
name = "my_benchmark"
harness = false

ysaito1001 commented 1 year ago

Thank you for providing us with a complete Cargo.toml. When I copied & pasted the contents of that file and ran cargo tree > tree.txt, I still failed to reproduce the issue of both hyper-tls v0.5.0 and hyper-rustls v0.23.2 appearing in tree.txt, i.e., I only saw hyper-rustls in the output:

├── aws-config v0.53.0
...
│   ├── aws-http v0.53.0
...
│   │   ├── aws-types v0.53.0
...
│   │   │   ├── aws-smithy-client v0.53.1
...
│   │   │   │   ├── hyper-rustls v0.23.2
...

I have two questions:

In your complete Cargo.toml, aws-smithy-client has version = "0.54.0" whereas aws-config and aws-types are version = 0.53.0. Is there a reason why you chose a different version for aws-smithy-client?
If you remove Cargo.lock and run cargo tree, do you still see both hyper-tls and hyper-rustls in the output?

lcmgh commented 1 year ago

Issue was on my side, sorry. Had added internal dependencies without default-features = false which by default used native-tls.

github-actions[bot] commented 1 year ago

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

awslabs / aws-sdk-rust