search

awslabs / aws-sdk-rust

AWS SDK for the Rust Programming Language
https://awslabs.github.io/aws-sdk-rust/
Apache License 2.0
3.01k stars 248 forks source link

Confusing `aws_config` behavior - FailedToLoadToken - 411 (Length Required) #997

Closed simbleau closed 10 months ago

simbleau commented 10 months ago

Describe the bug

When I run locally, with a ~/.aws/credentials file, everything works fine! When this runs on ubuntu-latest CI using environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, I get a very confusing error.

My hunch is that this is a symptom of a different issue.

The error

called `Result::unwrap()` on an `Err` value: Aws(Unhandled(Unhandled { source: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source: ProviderError(ProviderError { source: FailedToLoadToken(FailedToLoadToken { source: ServiceError(ServiceError { source: TokenError { kind: InvalidToken }, raw: Response { status: StatusCode(411), headers: Headers { headers: {"content-type": HeaderValue { _private: "text/html; charset=us-ascii" }, "server": HeaderValue { _private: "Microsoft-HTTPAPI/2.0" }, "date": HeaderValue { _private: "Fri, 08 Dec 2023 00:54:57 GMT" }, "connection": HeaderValue { _private: "close" }, "content-length": HeaderValue { _private: "344" }} }, body: SdkBody { inner: Once(Some(b"<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\"[http://www.w3.org/TR/html4/strict.dtd\](http://www.w3.org/TR/html4/strict.dtd/)">\r\n<HTML><HEAD><TITLE>Length Required</TITLE>\r\n<META HTTP-EQUIV=\"Content-Type\" Content=\"text/html; charset=us-ascii\"></HEAD>\r\n<BODY><h2>Length Required</h2>\r\n<hr><p>HTTP Error 411. The request must be chunked or have a content length.</p>\r\n</BODY></HTML>\r\n")), retryable: true }, extensions: Extensions } }) }) }), connection: Unknown } }), meta: ErrorMetadata { code: None, message: None, extras: None } }))

How I'm using it

// Init the client...
pub static DYNAMO_CLIENT: std::sync::LazyLock<aws_sdk_dynamodb::Client> =
    std::sync::LazyLock::new(|| {
        async_std::task::block_on(async {
            let aws_config =
                aws_config::defaults(aws_config::BehaviorVersion::latest())
                    .region("us-east-1") // Why does AWS_DEFAULT_REGION or AWS_REGION not work?
                    .load()
                    .await;
            aws_sdk_dynamodb::Client::new(&aws_config)
        })
    });

// Do something...
fn scan(
        limit: i32,
    ) -> impl Future<Output = Result<Vec<Self::Model>, DynamoError>> {
        async move {
            let items: Vec<HashMap<String, AttributeValue>> = DYNAMO_CLIENT
                .scan()
                .limit(limit)
                .table_name(Self::NAME)
                .send()
                .await?
                .items()
                .to_vec();

            let deserialized =
                serde_dynamo::from_items::<_, Self::Model>(items)?;

            Ok(deserialized)
        }
    }

Expected Behavior

N/A

Current Behavior

N/A

Reproduction Steps

N/A

Possible Solution

No response

Additional Information/Context

No response

Version

├── aws-config v1.0.3
│   ├── aws-credential-types v1.0.3
│   │   ├── aws-smithy-async v1.0.3
│   │   ├── aws-smithy-runtime-api v1.0.3
│   │   │   ├── aws-smithy-async v1.0.3 (*)
│   │   │   ├── aws-smithy-types v1.0.3
│   │   ├── aws-smithy-types v1.0.3 (*)
│   ├── aws-http v0.60.0
│   │   ├── aws-smithy-runtime-api v1.0.3 (*)
│   │   ├── aws-smithy-types v1.0.3 (*)
│   │   ├── aws-types v1.0.3
│   │   │   ├── aws-credential-types v1.0.3 (*)
│   │   │   ├── aws-smithy-async v1.0.3 (*)
│   │   │   ├── aws-smithy-runtime-api v1.0.3 (*)
│   │   │   ├── aws-smithy-types v1.0.3 (*)
│   ├── aws-runtime v1.0.3
│   │   ├── aws-credential-types v1.0.3 (*)
│   │   ├── aws-http v0.60.0 (*)
│   │   ├── aws-sigv4 v1.0.3
│   │   │   ├── aws-credential-types v1.0.3 (*)
│   │   │   ├── aws-smithy-http v0.60.0
│   │   │   │   ├── aws-smithy-runtime-api v1.0.3 (*)
│   │   │   │   ├── aws-smithy-types v1.0.3 (*)
│   │   │   ├── aws-smithy-runtime-api v1.0.3 (*)
│   │   │   ├── aws-smithy-types v1.0.3 (*)
│   │   ├── aws-smithy-async v1.0.3 (*)
│   │   ├── aws-smithy-http v0.60.0 (*)
│   │   ├── aws-smithy-runtime-api v1.0.3 (*)
│   │   ├── aws-smithy-types v1.0.3 (*)
│   │   ├── aws-types v1.0.3 (*)
│   ├── aws-sdk-sso v1.4.0
│   │   ├── aws-credential-types v1.0.3 (*)
│   │   ├── aws-http v0.60.0 (*)
│   │   ├── aws-runtime v1.0.3 (*)
│   │   ├── aws-smithy-async v1.0.3 (*)
│   │   ├── aws-smithy-http v0.60.0 (*)
│   │   ├── aws-smithy-json v0.60.0
│   │   │   └── aws-smithy-types v1.0.3 (*)
│   │   ├── aws-smithy-runtime v1.0.3
│   │   │   ├── aws-smithy-async v1.0.3 (*)
│   │   │   ├── aws-smithy-http v0.60.0 (*)
│   │   │   ├── aws-smithy-runtime-api v1.0.3 (*)
│   │   │   ├── aws-smithy-types v1.0.3 (*)
│   │   ├── aws-smithy-runtime-api v1.0.3 (*)
│   │   ├── aws-smithy-types v1.0.3 (*)
│   │   ├── aws-types v1.0.3 (*)
│   ├── aws-sdk-ssooidc v1.4.0
│   │   ├── aws-credential-types v1.0.3 (*)
│   │   ├── aws-http v0.60.0 (*)
│   │   ├── aws-runtime v1.0.3 (*)
│   │   ├── aws-smithy-async v1.0.3 (*)
│   │   ├── aws-smithy-http v0.60.0 (*)
│   │   ├── aws-smithy-json v0.60.0 (*)
│   │   ├── aws-smithy-runtime v1.0.3 (*)
│   │   ├── aws-smithy-runtime-api v1.0.3 (*)
│   │   ├── aws-smithy-types v1.0.3 (*)
│   │   ├── aws-types v1.0.3 (*)
│   ├── aws-sdk-sts v1.4.0
│   │   ├── aws-credential-types v1.0.3 (*)
│   │   ├── aws-http v0.60.0 (*)
│   │   ├── aws-runtime v1.0.3 (*)
│   │   ├── aws-smithy-async v1.0.3 (*)
│   │   ├── aws-smithy-http v0.60.0 (*)
│   │   ├── aws-smithy-json v0.60.0 (*)
│   │   ├── aws-smithy-query v0.60.0
│   │   │   ├── aws-smithy-types v1.0.3 (*)
│   │   ├── aws-smithy-runtime v1.0.3 (*)
│   │   ├── aws-smithy-runtime-api v1.0.3 (*)
│   │   ├── aws-smithy-types v1.0.3 (*)
│   │   ├── aws-smithy-xml v0.60.0
│   │   ├── aws-types v1.0.3 (*)
│   ├── aws-smithy-async v1.0.3 (*)
│   ├── aws-smithy-http v0.60.0 (*)
│   ├── aws-smithy-json v0.60.0 (*)
│   ├── aws-smithy-runtime v1.0.3 (*)
│   ├── aws-smithy-runtime-api v1.0.3 (*)
│   ├── aws-smithy-types v1.0.3 (*)
│   ├── aws-types v1.0.3 (*)

Environment details (OS name and version, etc.)

MacOS Ventura (my machine), ubuntu (CI machine)

Logs

No response

jdisanti commented 10 months ago

You mention ubuntu-latest in CI, so this is in GitHub Actions? The error you're getting is from the IMDS credentials provider, which is the very last one to be tried. That means the environment credentials provider already failed to find credentials in those env vars by the time it got to that error.

You can set up the tracing-subscriber crate with the env-filter feature, and initialize it with tracing_subscriber::fmt::init() in main. Doing so will allow you to run your program with RUST_LOG=debug to see what's going on.

I suspect the environment variables aren't actually getting set in the GitHub Actions workflow config.

simbleau commented 10 months ago

You mention ubuntu-latest in CI, so this is in GitHub Actions? The error you're getting is from the IMDS credentials provider, which is the very last one to be tried. That means the environment credentials provider already failed to find credentials in those env vars by the time it got to that error.

You can set up the tracing-subscriber crate with the env-filter feature, and initialize it with tracing_subscriber::fmt::init() in main. Doing so will allow you to run your program with RUST_LOG=debug to see what's going on.

I suspect the environment variables aren't actually getting set in the GitHub Actions workflow config.

Yes! it is GitHub actions.

Let me investigate further and report back. I think you're probably right.

simbleau commented 10 months ago

Logs

2023-12-08T03:09:48.277464Z DEBUG aws_config::fs_util: loaded home directory src="HOME"
2023-12-08T03:09:48.277601Z DEBUG load_config_file{file=Default(Config)}: aws_config::profile::parser::source: performing home directory substitution home="/home/runner" path="~/.aws/config"
2023-12-08T03:09:48.277655Z DEBUG load_config_file{file=Default(Config)}: aws_config::profile::parser::source: home directory expanded before="~/.aws/config" after="/home/runner/.aws/config"
2023-12-08T03:09:48.277696Z DEBUG load_config_file{file=Default(Config)}: aws_config::profile::parser::source: config file not found path=~/.aws/config
2023-12-08T03:09:48.277719Z DEBUG load_config_file{file=Default(Config)}: aws_config::profile::parser::source: config file loaded path=Some("/home/runner/.aws/config") size=0
2023-12-08T03:09:48.277790Z DEBUG load_config_file{file=Default(Credentials)}: aws_config::profile::parser::source: performing home directory substitution home="/home/runner" path="~/.aws/credentials"
2023-12-08T03:09:48.277807Z DEBUG load_config_file{file=Default(Credentials)}: aws_config::profile::parser::source: home directory expanded before="~/.aws/credentials" after="/home/runner/.aws/credentials"
2023-12-08T03:09:48.277825Z DEBUG load_config_file{file=Default(Credentials)}: aws_config::profile::parser::source: config file not found path=~/.aws/credentials
2023-12-08T03:09:48.277836Z DEBUG load_config_file{file=Default(Credentials)}: aws_config::profile::parser::source: config file loaded path=Some("/home/runner/.aws/credentials") size=0
2023-12-08T03:09:48.289494Z DEBUG build_profile_provider: hyper_rustls::config: with_native_roots processed 137 valid and 0 invalid certs    
2023-12-08T03:09:48.342353Z DEBUG invoke{service=dynamodb operation=Scan}:try_op: aws_smithy_runtime_api::client::interceptors::context: entering 'serialization' phase
2023-12-08T03:09:48.342537Z DEBUG invoke{service=dynamodb operation=Scan}:try_op: aws_smithy_runtime_api::client::interceptors::context: entering 'before transmit' phase
2023-12-08T03:09:48.342633Z DEBUG invoke{service=dynamodb operation=Scan}:try_op: aws_smithy_runtime::client::retries::strategy::standard: no client rate limiter configured, so no token is required for the initial request.
2023-12-08T03:09:48.342652Z DEBUG invoke{service=dynamodb operation=Scan}:try_op: aws_smithy_runtime::client::orchestrator: retry strategy has OKed initial request
2023-12-08T03:09:48.342680Z DEBUG invoke{service=dynamodb operation=Scan}:try_op: aws_smithy_runtime::client::orchestrator: beginning attempt #1
2023-12-08T03:09:48.342743Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt: aws_smithy_runtime::client::orchestrator::endpoints: resolving endpoint endpoint_params=EndpointResolverParams(TypeErasedBox[!Clone]:Params { region: Some("us-east-1"), use_dual_stack: false, use_fips: false, endpoint: None }) endpoint_prefix=None
2023-12-08T03:09:48.342800Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt: aws_smithy_runtime::client::orchestrator::endpoints: will use endpoint Endpoint { url: "https://dynamodb.us-east-1.amazonaws.com/", headers: {}, properties: {} }
2023-12-08T03:09:48.343113Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}: aws_config::meta::credentials::chain: provider in chain did not provide credentials provider=Environment context=the credential provider was not enabled: environment variable not set (CredentialsNotLoaded(CredentialsNotLoaded { source: "environment variable not set" }))
2023-12-08T03:09:48.343187Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}: aws_config::meta::credentials::chain: provider in chain did not provide credentials provider=Profile context=the credential provider was not enabled: No profiles were defined (CredentialsNotLoaded(CredentialsNotLoaded { source: NoProfilesDefined }))
2023-12-08T03:09:48.343234Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}: aws_config::meta::credentials::chain: provider in chain did not provide credentials provider=WebIdentityToken context=the credential provider was not enabled: $AWS_WEB_IDENTITY_TOKEN_FILE was not set (CredentialsNotLoaded(CredentialsNotLoaded { source: "$AWS_WEB_IDENTITY_TOKEN_FILE was not set" }))
2023-12-08T03:09:48.343291Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}: aws_config::meta::credentials::chain: provider in chain did not provide credentials provider=EcsContainer context=the credential provider was not enabled: ECS provider not configured (CredentialsNotLoaded(CredentialsNotLoaded { source: "ECS provider not configured" }))
2023-12-08T03:09:48.343328Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}: aws_config::imds::credentials: loading credentials from IMDS
2023-12-08T03:09:48.343519Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op: aws_smithy_runtime_api::client::interceptors::context: entering 'serialization' phase
2023-12-08T03:09:48.343600Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op: aws_smithy_runtime_api::client::interceptors::context: entering 'before transmit' phase
2023-12-08T03:09:48.343643Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op: aws_smithy_runtime::client::retries::strategy::standard: no client rate limiter configured, so no token is required for the initial request.
2023-12-08T03:09:48.343669Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op: aws_smithy_runtime::client::orchestrator: retry strategy has OKed initial request
2023-12-08T03:09:48.343702Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op: aws_smithy_runtime::client::orchestrator: beginning attempt #1
2023-12-08T03:09:48.343752Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt: aws_smithy_runtime::client::orchestrator::endpoints: resolving endpoint endpoint_params=EndpointResolverParams(TypeErasedBox[!Clone]:()) endpoint_prefix=None
2023-12-08T03:09:48.343808Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt: aws_smithy_runtime::client::orchestrator::endpoints: will use endpoint Endpoint { url: "http://169.254.169.254/", headers: {}, properties: {} }
2023-12-08T03:09:48.343925Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt: aws_config::imds::client::token: IMDS token cache miss
2023-12-08T03:09:48.344127Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt:invoke{service=imds operation=get-token}:try_op: aws_smithy_runtime_api::client::interceptors::context: entering 'serialization' phase
2023-12-08T03:09:48.344238Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt:invoke{service=imds operation=get-token}:try_op: aws_smithy_runtime_api::client::interceptors::context: entering 'before transmit' phase
2023-12-08T03:09:48.344286Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt:invoke{service=imds operation=get-token}:try_op: aws_smithy_runtime::client::retries::strategy::standard: no client rate limiter configured, so no token is required for the initial request.
2023-12-08T03:09:48.344315Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt:invoke{service=imds operation=get-token}:try_op: aws_smithy_runtime::client::orchestrator: retry strategy has OKed initial request
2023-12-08T03:09:48.344353Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt:invoke{service=imds operation=get-token}:try_op: aws_smithy_runtime::client::orchestrator: beginning attempt #1
2023-12-08T03:09:48.344406Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt:invoke{service=imds operation=get-token}:try_op:try_attempt: aws_smithy_runtime::client::orchestrator::endpoints: resolving endpoint endpoint_params=EndpointResolverParams(TypeErasedBox[!Clone]:()) endpoint_prefix=None
2023-12-08T03:09:48.344464Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt:invoke{service=imds operation=get-token}:try_op:try_attempt: aws_smithy_runtime::client::orchestrator::endpoints: will use endpoint Endpoint { url: "http://169.254.169.254/", headers: {}, properties: {} }
2023-12-08T03:09:48.344592Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt:invoke{service=imds operation=get-token}:try_op:try_attempt: aws_smithy_runtime_api::client::interceptors::context: entering 'transmit' phase
2023-12-08T03:09:48.344829Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt:invoke{service=imds operation=get-token}:try_op:try_attempt: hyper::client::connect::http: connecting to 169.254.169.254:80
2023-12-08T03:09:48.345849Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt:invoke{service=imds operation=get-token}:try_op:try_attempt: hyper::client::connect::http: connected to 169.254.169.254:80
2023-12-08T03:09:48.346255Z DEBUG hyper::proto::h1::io: flushed 255 bytes
2023-12-08T03:09:48.346559Z DEBUG hyper::proto::h1::io: parsed 5 headers
2023-12-08T03:09:48.346573Z DEBUG hyper::proto::h1::conn: incoming body is content-length (344 bytes)
2023-12-08T03:09:48.346604Z DEBUG hyper::proto::h1::conn: incoming body completed
2023-12-08T03:09:48.346777Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt:invoke{service=imds operation=get-token}:try_op:try_attempt: aws_smithy_runtime_api::client::interceptors::context: entering 'before deserialization' phase
2023-12-08T03:09:48.346841Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt:invoke{service=imds operation=get-token}:try_op:try_attempt: aws_smithy_runtime_api::client::interceptors::context: entering 'deserialization' phase
2023-12-08T03:09:48.346993Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt:invoke{service=imds operation=get-token}:try_op:try_attempt: aws_smithy_runtime_api::client::interceptors::context: entering 'after deserialization' phase
2023-12-08T03:09:48.347090Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt:invoke{service=imds operation=get-token}:try_op: aws_smithy_runtime::client::retries::strategy::standard: encountered unretryable error attempts=1 max_attempts=4
2023-12-08T03:09:48.347126Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt:invoke{service=imds operation=get-token}:try_op: aws_smithy_runtime::client::orchestrator: a retry is either unnecessary or not possible, exiting attempt loop
2023-12-08T03:09:48.347231Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op:try_attempt: aws_smithy_runtime::client::orchestrator: encountered orchestrator error; halting
2023-12-08T03:09:48.347303Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op: aws_smithy_runtime::client::retries::strategy::standard: encountered unretryable error attempts=1 max_attempts=4
2023-12-08T03:09:48.347332Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}:load_credentials{provider=Ec2InstanceMetadata}:invoke{service=imds operation=get}:try_op: aws_smithy_runtime::client::orchestrator: a retry is either unnecessary or not possible, exiting attempt loop
2023-12-08T03:09:48.347428Z  WARN invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}: aws_config::meta::credentials::chain: provider failed to provide credentials provider=Ec2InstanceMetadata error=an error occurred while loading credentials: failed to load IMDS session token: service error: invalid token (ProviderError(ProviderError { source: FailedToLoadToken(FailedToLoadToken { source: ServiceError(ServiceError { source: TokenError { kind: InvalidToken }, raw: Response { status: StatusCode(411), headers: Headers { headers: {"content-type": HeaderValue { _private: "text/html; charset=us-ascii" }, "server": HeaderValue { _private: "Microsoft-HTTPAPI/2.0" }, "date": HeaderValue { _private: "Fri, 08 Dec 2023 03:09:47 GMT" }, "connection": HeaderValue { _private: "close" }, "content-length": HeaderValue { _private: "344" }} }, body: SdkBody { inner: Once(Some(b"<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\"[http://www.w3.org/TR/html4/strict.dtd\](http://www.w3.org/TR/html4/strict.dtd/)">\r\n<HTML><HEAD><TITLE>Length Required</TITLE>\r\n<META HTTP-EQUIV=\"Content-Type\" Content=\"text/html; charset=us-ascii\"></HEAD>\r\n<BODY><h2>Length Required</h2>\r\n<hr><p>HTTP Error 411. The request must be chunked or have a content length.</p>\r\n</BODY></HTML>\r\n")), retryable: true }, extensions: Extensions } }) }) }))
simbleau commented 10 months ago

@jdisanti - Appears as though due to not seeing the environment variables, 

2023-12-08T03:09:48.360329Z DEBUG invoke{service=dynamodb operation=Scan}:try_op:try_attempt:lazy_load_identity:provide_credentials{provider=default_chain}: aws_config::meta::credentials::chain: provider in chain did not provide credentials provider=Environment context=the credential provider was not enabled: environment variable not set (CredentialsNotLoaded(CredentialsNotLoaded { source: "environment variable not set" }))

However this is false... The environment variables are definitely there and correct.

image
jdisanti commented 10 months ago

Secrets in GitHub Actions aren't environment variables. There's a little extra piping you have to do in the workflow config, for example: https://github.com/smithy-lang/smithy-rs/blob/main/.github/workflows/ci.yml#L34-L35

github-actions[bot] commented 10 months ago

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.

simbleau commented 10 months ago

Perfect! Thanks

github-actions[bot] commented 10 months ago

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.