Bad types after smithy upgrade

[xconverge]

Checkboxes for prior research

• [X] I've gone through Developer Guide and API reference
• [X] I've checked AWS Forums and StackOverflow.
• [X] I've searched for previous similar issues and didn't find any solution.

Describe the bug

Here are 2 examples of some issues I am experiencing with types with 3.363.0

I scanned through https://github.com/aws/aws-sdk-js-v3/blob/main/CHANGELOG.md and can't find anything that makes these seem like intentional changes, sorry if this is all a bit vague but maybe someone with more information can ask more pointed questions/add something that helps.

3.362.0 is all good, 3.363.0 has all sorts of issues. The only change that seems relevant to me is https://github.com/aws/aws-sdk-js-v3/pull/4873

Issues I see with @aws-sdk/s3-request-presigner:

Argument of type 'S3' is not assignable to parameter of type 'Client<any, ServiceInputTypes, MetadataBearer, any>'.
  The types of 'middlewareStack.add' are incompatible between these types.
    Type '{ (middleware: InitializeMiddleware<ServiceInputTypes, ServiceOutputTypes>, options?: (InitializeHandlerOptions & AbsoluteLocation) | undefined): void; (middleware: SerializeMiddleware<...>, options: SerializeHandlerOptions & AbsoluteLocation): void; (middleware: BuildMiddleware<...>, options: BuildHandlerOptions & ...' is not assignable to type '{ (middleware: InitializeMiddleware<ServiceInputTypes, MetadataBearer>, options?: (InitializeHandlerOptions & AbsoluteLocation) | undefined): void; (middleware: SerializeMiddleware<...>, options: SerializeHandlerOptions & AbsoluteLocation): void; (middleware: BuildMiddleware<...>, options: BuildHandlerOptions & Abso...'.
      Types of parameters 'middleware' and 'middleware' are incompatible.
        Types of parameters 'next' and 'next' are incompatible.
          Type 'InitializeHandler<ServiceInputTypes, ServiceOutputTypes>' is not assignable to type 'InitializeHandler<ServiceInputTypes, MetadataBearer>'.
            Type 'ServiceOutputTypes' is not assignable to type 'MetadataBearer'.ts(2345)

Issues I see with @aws-sdk/client-s3:

Property 'Body' does not exist on type 'GetObjectCommandOutput'.ts(2339)

SDK version number

@aws-sdk/s3-request-presigner@3.363.0

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

node v18.16.0

Reproduction Steps

Code snippets in screenshots are one example

Observed Behavior

Lots of typescript errors and weirdness with 3.363.0

Expected Behavior

Types work correctly like in 3.362.0

Possible Solution

No response

Additional Information/Context

No response

[xconverge]

I also just noticed this was reported while I was reporting my issue, and it looks related IMO #4927

[RanVaknin]

Hi @xconverge ,

Thanks for opening this issue.

The code snippets you attached as images are incomplete and hard to assess since they are in image form.

I wrote some repro code loosly based on the information you have provided:

import { S3Client, GetObjectCommand, HeadObjectCommand, GetObjectCommandOutput } from "@aws-sdk/client-s3";

const main = async () => {
  const client = new S3Client({ region: "us-east-1" });
  const params = { Bucket: "testbucket", Key: "hey.txt" };
  const headCommand = new HeadObjectCommand(params);

  return client.send(headCommand).then(async () => {
    const getCommand = new GetObjectCommand(params);
    const data: GetObjectCommandOutput = await client.send(getCommand);

    if (data.Body) {
      console.log("Body exists on data");
      return;
    }
    console.log("Body does not exist on data");
  });
};

main()
  .catch(console.error);

This compiles and runs successfully:

$ npm list
4928@1.0.0 /Users/rvaknin/test_folder/4928
├── @aws-sdk/client-s3@3.363.0
└── @types/node@20.4.0
$ tsc src/sample.ts 
$ node src/sample.js
Body exists on data

This is odd, because I was not able to reproduce #4927 either. To give you an idea of how I'm reproducing this, with each reproduction I'm creating a new project from scratch, so Im wondering maybe your code has some dependencies that aren't resolving correctly. Can you deleting your node_modules and package-lock.json and try running npm install again to see if it solves the problem?

Otherwise, if you can upload a minimal github repo that can consistently reproduce this behavior, it would be extremely helpful.

Thanks again, Ran~

[xconverge]

@RanVaknin I did not use client.send, I used the alternative API, so perhaps that makes a difference and is why your repro code doesnt repro it. Sorry for the images, I knew it was a half-baked bug report but was hoping there was something obvious.

If instead of what you wrote you were to do something like this, what do you see?

import { S3Client, GetObjectCommand, HeadObjectCommand, GetObjectCommandOutput } from "@aws-sdk/client-s3";

const main = async () => {
  const client = new S3Client({ region: "us-east-1" });
  const params = { Bucket: "testbucket", Key: "hey.txt" };

  return client.send(headCommand).then(async () => {
    //const getCommand = new GetObjectCommand(params);
    const data: GetObjectCommandOutput = await client.getObject(params);

    if (data.Body) {
      console.log("Body exists on data");
      return;
    }
    console.log("Body does not exist on data");
  });
};

main()
  .catch(console.error);

I will try to get to an actual small reproducible example tomorrow, but I experienced this over a handful of repositories and all of it looks similar (albeit slight differences between which client/lib I was using)

[Joel-PeakMetrics]

I'm seeing the same problem with deno (strict TS checks). Deno version 1.34.2

import { GetObjectCommand, PutObjectCommand, S3Client } from "npm:@aws-sdk/client-s3@3.363.0";
const s3Client = new S3Client({
    "region":"us-east-1",
    "endpoint":"http://localhost:4566",
    "forcePathStyle": true
})

const command = new GetObjectCommand({
    Bucket: "local-raw-data",
    Key: "*",
});
const objectsFound = await s3Client.send(command);

$ deno check s3test.ts
Check file:///.../s3test.ts
TS2345 [ERROR]: Argument of type 'GetObjectCommand' is not assignable to parameter of type 'Command<ServiceInputTypes, GetObjectCommandInput, ServiceOutputTypes, GetObjectCommandOutput, SmithyResolvedConfiguration<...>>'.
  Types of property 'resolveMiddleware' are incompatible.
    Type '(clientStack: MiddlewareStack<ServiceInputTypes, ServiceOutputTypes>, configuration: S3ClientResolvedConfig, options?: HttpHandlerOptions | undefined) => Handler<...>' is not assignable to type '(stack: MiddlewareStack<ServiceInputTypes, ServiceOutputTypes>, configuration: SmithyResolvedConfiguration<HttpHandlerOptions>, options: any) => Handler<...>'.
      Types of parameters 'clientStack' and 'stack' are incompatible.
        Type 'import("file:///.../node_modules/.deno/@smithy+smithy-client@1.0.3/node_modules/@smithy/types/dist-types/middleware.d.ts").MiddlewareStack<import("file:///.../node_modules/.deno/@aws-sdk+client-s3@3.363.0/node_mod...' is not assignable to type 'import("file:///.../node_modules/.deno/@aws-sdk+client-s3@3.363.0/node_modules/@smithy/types/dist-types/middleware.d.ts").MiddlewareStack<import("file:///.../node_modules/.deno/@aws-sdk+client-s3@3.363.0/node_modu...'.
          Types of property 'concat' are incompatible.
            Type '<InputType extends import("file:///.../node_modules/.deno/@aws-sdk+client-s3@3.363.0/node_modules/@aws-sdk/client-s3/dist-types/S3Client.d.ts").ServiceInputTypes, OutputType extends import("file:///.../node_module...' is not assignable to type '<InputType extends import("file:///.../node_modules/.deno/@aws-sdk+client-s3@3.363.0/node_modules/@aws-sdk/client-s3/dist-types/S3Client.d.ts").ServiceInputTypes, OutputType extends import("file:///.../node_module...'. Two different types with this name exist, but they are unrelated.
              Types of parameters 'from' and 'from' are incompatible.
                Type 'MiddlewareStack<InputType, OutputType>' is not assignable to type 'MiddlewareStack<InputType, ServiceOutputTypes>'.
                  Types of property 'addRelativeTo' are incompatible.
                    Type '(middleware: MiddlewareType<InputType, OutputType>, options: RelativeMiddlewareOptions) => void' is not assignable to type '(middleware: MiddlewareType<InputType, ServiceOutputTypes>, options: RelativeMiddlewareOptions) => void'.
                      Types of parameters 'middleware' and 'middleware' are incompatible.
                        Type 'MiddlewareType<InputType, ServiceOutputTypes>' is not assignable to type 'MiddlewareType<InputType, OutputType>'.
                          Type 'InitializeMiddleware<InputType, ServiceOutputTypes>' is not assignable to type 'MiddlewareType<InputType, OutputType>'.
                            Type 'InitializeMiddleware<InputType, ServiceOutputTypes>' is not assignable to type 'InitializeMiddleware<InputType, OutputType>'.
                              Call signature return types 'InitializeHandler<InputType, ServiceOutputTypes>' and 'InitializeHandler<InputType, OutputType>' are incompatible.
                                Type 'Promise<InitializeHandlerOutput<ServiceOutputTypes>>' is not assignable to type 'Promise<InitializeHandlerOutput<OutputType>>'.
                                  Type 'InitializeHandlerOutput<ServiceOutputTypes>' is not assignable to type 'InitializeHandlerOutput<OutputType>'.
                                    Types of property 'output' are incompatible.
                                      Type 'ServiceOutputTypes' is not assignable to type 'OutputType'.
                                        'ServiceOutputTypes' is assignable to the constraint of type 'OutputType', but 'OutputType' could be instantiated with a different subtype of constraint 'ServiceOutputTypes'.
                                          Type 'AbortMultipartUploadCommandOutput' is not assignable to type 'OutputType'.
                                            'AbortMultipartUploadCommandOutput' is assignable to the constraint of type 'OutputType', but 'OutputType' could be instantiated with a different subtype of constraint 'ServiceOutputTypes'.
const objectsFound = await s3Client.send(command);
                                         ~~~~~~~
    at file:///.../s3test.ts:14:42

Found 2 errors.

[RanVaknin]

Hi @xconverge,

I noticed some inconsistencies in your code. It seems like you were trying to use S3.headObject() rather than s3Client.send(new HeadObjectCommand({})) but I think I understand what you were trying to demonstrate:

import { S3, GetObjectCommandOutput } from "@aws-sdk/client-s3";

const main = async () => {
  const client = new S3({ region: "us-east-1" });
  const params = { Bucket: "testbucket", Key: "hey.txt" };

  return client.headObject(params).then(async () => {
    const data: GetObjectCommandOutput = await client.getObject(params);

    if (data.Body) {
      console.log("Body exists on data");
      return;
    }
    console.log("Body does not exist on data");
  });
};

main()
  .catch(console.error);

Yet, this modification doesn't seem to alter the outcome. The body is still present, which aligns with the type definitions.

We are aware that some users are encountering this issue, but it's been challenging for our team to replicate. If you could provide a minimal repository demonstrating this issue, it would be greatly appreciated.

Thanks again, Ran~

[xconverge]

Deleting my lock file and reinstalling seems to resolve the issues...

[richgarner-nsw-gov]

I think this is the same as I raised here

Especially the comment

[frontendphil]

I see a very similar error. The s3Client method returns new S3Client and fills in the AWS region. Key and Bucket are both strings.

We're running on TS 5.0.4 right now if that helps.

[Naddiseo]

Deleting my lock file and reinstalling seems to resolve the issues...

This also seems to work for me, which seems to indicate a peer, or deep, dependency mismatch?

I dug a little bit deeper into the yarn.lock file and experimented. Here's my starting package.json wrt @aws-sdk:

{
   "dependencies": {
    "@aws-sdk/client-s3": "^3.362.0",
    "@aws-sdk/client-sns": "^3.362.0",
    "@aws-sdk/s3-request-presigner": "^3.362.0",
   "@aws-sdk/client-secrets-manager": "^3.362.0",
   }
}

and after upgrade

{
   "dependencies": {
   "@aws-sdk/client-s3": "^3.367.0",
    "@aws-sdk/client-sns": "^3.363.0",
    "@aws-sdk/s3-request-presigner": "^3.367.0",
    "@aws-sdk/client-secrets-manager": "^3.363.0",
   }
}

I looked at my yarn.lock file at the "@aws-sdk/*" and "@smithy*" related packages to compare, the output is in a gist, It has 4 files:

1. The yarn.lock before the upgrade (works)
2. yarn.lock after upgrading via yarn upgrade-interactive (does not work)
3. yarn.lock after removing all @aws-sdk and @smithy entries, then upgrading (DOES WORK)
4. The diff between files 2 and 3

here's the diff for quick reference:

726c726
< "@aws-sdk/types@npm:3.357.0":
---
> "@aws-sdk/types@npm:3.357.0, @aws-sdk/types@npm:^3.222.0":
735,743d734
< "@aws-sdk/types@npm:^3.222.0":
<   version: 3.257.0
<   resolution: "@aws-sdk/types@npm:3.257.0"
<   dependencies:
<     tslib: ^2.3.1
<   checksum: eab2154f7d27d0bec29df41374481652525e1658ad987c64ed02cbe7b7388c563e6a8d317c8fc564ac6fbd97c98be700a29352c9acebc3903cf3305431bf0a86
<   languageName: node
<   linkType: hard
< 
796,797c787,788
<   version: 3.55.0
<   resolution: "@aws-sdk/util-locate-window@npm:3.55.0"
---
>   version: 3.310.0
>   resolution: "@aws-sdk/util-locate-window@npm:3.310.0"
799,800c790,791
<     tslib: ^2.3.1
<   checksum: 5bc79779a17ae4dd5cbcc221df6e85c976a149c0868745ae58075a3c51506aeeb087dfde4b3e5179004461243b11f3a458de720ee92866c6a1b65182326259ae
---
>     tslib: ^2.5.0
>   checksum: d552ce5f0f836ecb13d7920ae650552c56706f26a5e8abf894ba471e18775a3791869bda95269153735bac9d211efc3ba78ea01c34428c3fed4318ac693a08bc
834,835c825,826
<   version: 3.109.0
<   resolution: "@aws-sdk/util-utf8-browser@npm:3.109.0"
---
>   version: 3.259.0
>   resolution: "@aws-sdk/util-utf8-browser@npm:3.259.0"
838c829
<   checksum: 8311763b04261dab5995ec67abf31795f41e9c4b1ad635ed305735e14c7e3bc48e9ae349a06aab485390358a6a58e97190144ea51190983cec4ae665887b219b
---
>   checksum: b6a1e580da1c9b62c749814182a7649a748ca4253edb4063aa521df97d25b76eae3359eb1680b86f71aac668e05cc05c514379bca39ebf4ba998ae4348412da8
1089,1099c1080
< "@smithy/protocol-http@npm:^1.0.1":
<   version: 1.0.1
<   resolution: "@smithy/protocol-http@npm:1.0.1"
<   dependencies:
<     "@smithy/types": ^1.0.0
<     tslib: ^2.5.0
<   checksum: ba9ac4880fed48eeea0813663c94c765fe5b900f2fdac4f5de6524306bbc6645829f48bc175d202076b83acaccf008ed77f4b5546a4c180315f253e22fe6c89f
<   languageName: node
<   linkType: hard
< 
< "@smithy/protocol-http@npm:^1.1.0, @smithy/protocol-http@npm:^1.1.1":
---
> "@smithy/protocol-http@npm:^1.0.1, @smithy/protocol-http@npm:^1.1.0, @smithy/protocol-http@npm:^1.1.1":
1175,1184c1156
< "@smithy/types@npm:^1.0.0":
<   version: 1.0.0
<   resolution: "@smithy/types@npm:1.0.0"
<   dependencies:
<     tslib: ^2.5.0
<   checksum: ec05163564af050088f3c21cb047640ca842bea645c2a73624475b486d5df8ad9c494bf683a498f4b467b84fab2817cc199893dfb5cee30dce1e0172ab38db00
<   languageName: node
<   linkType: hard
< 
< "@smithy/types@npm:^1.1.0, @smithy/types@npm:^1.1.1":
---
> "@smithy/types@npm:^1.0.0, @smithy/types@npm:^1.1.0, @smithy/types@npm:^1.1.1":

From my reading of it, without removing the yarn.lock first, there seems to be more versions of @smithy/types and @aws-sdk/types hanging around in the working version

[benj-dobs]

I also had this issue and was only able to resolve it by deleting yarn.lock and regenerating it from scratch. I tried at first just deleting the top-level @aws-sdk/* and @smithy/* entries, but this was not successful.

My type errors indicate that there are two incompatible versions of @smithy/types installed, one at the root of node_modules and one inside @smithy/signature-v4. Here's an example of a type error when using client-cognito-identity-provider

Argument of type 'VerifyUserAttributeCommand' is not assignable to parameter of type 'Command<ServiceInputTypes, VerifyUserAttributeCommandInput, ServiceOutputTypes, VerifyUserAttributeCommandOutput, SmithyResolvedConfiguration<...>>'.
  Types of property 'resolveMiddleware' are incompatible.
    Type '(clientStack: MiddlewareStack<ServiceInputTypes, ServiceOutputTypes>, configuration: CognitoIdentityProviderClientResolvedConfig, options?: HttpHandlerOptions | undefined) => Handler<...>' is not assignable to type '(stack: MiddlewareStack<ServiceInputTypes, ServiceOutputTypes>, configuration: SmithyResolvedConfiguration<HttpHandlerOptions>, options: any) => Handler<...>'.
      Types of parameters 'clientStack' and 'stack' are incompatible.
        Type 'import("<PROJECT_ROOT>/node_modules/@smithy/signature-v4/node_modules/@smithy/types/dist-types/middleware").MiddlewareStack<import("<PROJECT_ROOT>/node_modules/@aws-sdk/client-cognito-identity-provider/dist-types/CognitoIdentityProviderC...' is not assignable to type 'import("<PROJECT_ROOT>/node_modules/@smithy/types/dist-types/middleware").MiddlewareStack<import("<PROJECT_ROOT>/node_modules/@aws-sdk/client-cognito-identity-provider/dist-types/CognitoIdentityProviderClient").ServiceInputTypes, import(...'.

[xconverge]

What is the latest on this @RanVaknin ?

It got quite a few thumbs-ups, so I was obviously not alone, I have resolved it for myself by deleting the lock file, but I imagine you want to close the issue?

[switz]

Seeing this error – deleting lockfiles shouldn't be a solution (nor does it fix it for me).

[RanVaknin]

Hi to all the folks still running into this issue. Can you please update your SDK dependencies and see if the issue is still present?

Thanks, Ran~

[github-actions[bot]]

This issue has not received a response in 1 week. If you still think there is a problem, please leave a comment to avoid the issue from automatically closing.

[Joel-PeakMetrics]

Hello @RanVaknin, I upgraded to AWS SDK v3.379.1, but I am still seeing the type check issue.

[xconverge]

I just had to delete and recreate my lockfile again to update

[frankleng]

issue went away after forcing @smithy/types : 2.2.2 s3 was using 2.0.2 according to lock file

[oscarhermoso]

Ditto - I also resolved this issue by installing @smithy/types=@2.3.0 instead of relying on nested dependendencies in the lock file.

[shellscape]

Just ran across this with a Property 'TopicArn' does not exist on type 'ServiceOutputTypes' using CreateTopicCommand from @aws-sdk/client-sns. My repo is a monorepo with pnpm, and another project in the monorepo is using an older version of aws-sdk (the context is valid in this case). Installing @smithy/types@latest directly in the affected project solved the errors as well.

[Gkleinereva]

I ran into this issue on @aws-sdk/client-s3@3.465.0. I'm using pnpm - re-generating the lock file and reinstalling/rebuilding node_modules fixed the issue for me

[anna-kol]

it happened to me in the newest version 3.465.0

it thinks it should be "new S3Client([{ region: 'us-east1' }])", which throws region not found error instead of "new S3Client({ region: 'us-east1' })" which is correct but the typescript doesn't accept

i tried to delete node_modules and the lock file like suggested, and also install the latest smithy/types 2.6.0 instead of relying on the dependency and it didn't help

reverting to "@aws-sdk/client-s3": "3.13.1" helped.

[espetro]

The issue with getSignedUrl still happens! I'm using the following versions:

{
  "@aws-sdk/client-s3": "3.540.0",
  "@aws-sdk/s3-request-presigner": "3.540.0",
}

[letharion]

For what it's worth I ended up here after googling for a similar error. rm yarn.lock && yarn resolved the issue for me. The git diff after that is gigantic, >16000 LOC, so I won't post that here, but the parts I'm assuming are relevant to this are a large number of minor upgrades in the form of:

-    "@aws-sdk/client-s3": "^3.554.0",
+    "@aws-sdk/client-s3": "^3.621.0",

and

-    "@smithy/middleware-content-length": "npm:^3.0.4"
-    "@smithy/middleware-endpoint": "npm:^3.0.5"
-    "@smithy/middleware-retry": "npm:^3.0.10"
+    "@smithy/middleware-content-length": "npm:^3.0.5"
+    "@smithy/middleware-endpoint": "npm:^3.1.0"
+    "@smithy/middleware-retry": "npm:^3.0.13"

[RanVaknin]

Hi everyone on the thread. It seems like there is a mix of people that solved this by deleting the lock file and npm installing the latest versions of the SDK. For others it was to install a newer version of smithy types directly, for others its a different issue altogether.

Because this was not reproducible, and people are chiming in with different reasoning, we are not able to make actionable suggestions. I ask that if you are still affected to do the following:

1. Delete the lock file and re-install your dependencies - preferably use the latest version
2. If the issue persists please open a new issue, and provide a small github repository that we can clone and test locally ourselves to better help you.

Thanks, Ran~

[shellscape]

It's fairly irresponsible to recommend deleting a lock file to correct a single dependency issue. they're not supposed to be superfluous like that.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.