Use ^3.0.0 versions of client-generated dependencies

marcogrcr commented 2 years ago

Clients generated with version 0.12.0 of smithy-types use versions 3.17x.0 of @aws-sdk/*.

Usage of static version numbers (e.g. { "@aws-sdk/types": "3.171.0" }) instead of dynamic version numbers (e.g. { "@aws-sdk/types": "^3.171.0" }), causes TypeScript compilation errors if a package.json depends on:

A package created with smithy-typescript, and
A @aws-sdk/* package with a newer version than the ones used by smithy-typescript.

Is it possible to modify TypeScriptDependency.java so that auto-generated clients use ^3.0.0 instead of static versions?

To illustrate the problem, take the following example where we simulate that @aws-sdk/client-s3 is created with smithy-typescript@0.11.0 (i.e. it uses @aws-sdk/types@3.55.0) and @aws-sdk/client-dynamodb is created with smithy-typescript@0.12.0 (i.e. it uses @aws-sdk/types@3.171.0):

node --version: v14.20.0
npm --version: 6.14.17

package.json

{
  "name": "proof-of-concept",
  "version": "1.0.0",
  "dependencies": {
    "@aws-sdk/client-dynamodb": "3.172.0",
    "@aws-sdk/client-s3": "3.58.0"
  },
  "devDependencies": {
    "@types/node": "14.18.27",
    "typescript": "4.6.4"
  }
}

This results in the following node_modules/ folder:

node_modules/
- @aws-sdk/
- client-dynamodb/ (3.171.0)
- client-s3/ (3.55.0)
  - node_modules/
  - @aws-sdk/
    - types/ (3.55.0)
    - ...
- types/ (3.171.0)
- ...
- ...

src/main.ts

import { DynamoDBClient } from "@aws-sdk/client-dynamodb"; // @aws-sdk/types 3.171.0
import { S3Client } from "@aws-sdk/client-s3"; // @aws-sdk/types 3.55.0
import { Client, MetadataBearer } from "@aws-sdk/types"; // @aws-sdk/types 3.171.0

const c1: Client<object, MetadataBearer, unknown> = new DynamoDBClient({});
const c2: Client<object, MetadataBearer, unknown> = new S3Client({});

When compiled with:

npx tsc src/main.ts --strictNullChecks

You get:

src/main.ts:6:7 - error TS2322: Type 'S3Client' is not assignable to type 'Client<object, MetadataBearer, unknown>'.
  The types of 'middlewareStack.concat' are incompatible between these types.
    Type '<InputType extends ServiceInputTypes, OutputType extends ServiceOutputTypes>(from: MiddlewareStack<InputType, OutputType>) => MiddlewareStack<...>' is not assignable to type '<InputType extends object, OutputType extends MetadataBearer>(from: MiddlewareStack<InputType, OutputType>) => MiddlewareStack<InputType, OutputType>'.
      Types of parameters 'from' and 'from' are incompatible.
        Type 'MiddlewareStack<InputType, OutputType>' is not assignable to type 'MiddlewareStack<InputType, ServiceOutputTypes>'.
          Types of property 'concat' are incompatible.
            Type '<InputType extends InputType, OutputType extends OutputType>(from: MiddlewareStack<InputType, OutputType>) => MiddlewareStack<InputType, OutputType>' is not assignable to type '<InputType extends InputType, OutputType extends ServiceOutputTypes>(from: MiddlewareStack<InputType, OutputType>) => MiddlewareStack<InputType, OutputType>'.
              Types of parameters 'from' and 'from' are incompatible.
                Type 'import("/.../node_modules/@aws-sdk/client-s3/node_modules/@aws-sdk/types/dist-types/middleware").MiddlewareStack<InputType, OutputType>' is not assignable to type 'import("/.../node_modules/@aws-sdk/types/dist-types/middleware").MiddlewareStack<InputType, OutputType>'.
                  Types of property 'add' are incompatible.
                    Type '{ (middleware: import("/.../node_modules/@aws-sdk/client-s3/node_modules/@aws-sdk/types/dist-types/middleware").InitializeMiddleware<InputType, OutputType>, options?: (import("/...' is not assignable to type '{ (middleware: import("/.../node_modules/@aws-sdk/types/dist-types/middleware").InitializeMiddleware<InputType, OutputType>, options?: (import("/.../node_modules/@aws-sd...'.
                      Types of parameters 'middleware' and 'middleware' are incompatible.
                        Types of parameters 'next' and 'next' are incompatible.
                          Type 'import("/.../node_modules/@aws-sdk/client-s3/node_modules/@aws-sdk/types/dist-types/middleware").InitializeHandler<InputType, OutputType>' is not assignable to type 'import("/.../node_modules/@aws-sdk/types/dist-types/middleware").InitializeHandler<InputType, OutputType>'.
                            Type 'Promise<import("/.../node_modules/@aws-sdk/client-s3/node_modules/@aws-sdk/types/dist-types/middleware").InitializeHandlerOutput<OutputType>>' is not assignable to type 'Promise<import("/.../node_modules/@aws-sdk/types/dist-types/middleware").InitializeHandlerOutput<OutputType>>'.
                              Type 'import("/.../node_modules/@aws-sdk/client-s3/node_modules/@aws-sdk/types/dist-types/middleware").InitializeHandlerOutput<OutputType>' is not assignable to type 'import("/.../node_modules/@aws-sdk/types/dist-types/middleware").InitializeHandlerOutput<OutputType>'.
                                Types of property 'output' are incompatible.
                                  Type 'OutputType' is not assignable to type 'OutputType'. Two different types with this name exist, but they are unrelated.
                                    'OutputType' is assignable to the constraint of type 'OutputType', but 'OutputType' could be instantiated with a different subtype of constraint 'MetadataBearer'.

6 const c2: Client<object, MetadataBearer, unknown> = new S3Client({});

Found 1 error in src/main.ts:6

marcogrcr commented 2 years ago

For folks affected by this issue, in the meantime, you can workaround by using npm@^8.3.0 and configure the overrides option:

package.json

{
  "name": "proof-of-concept",
  "version": "1.0.0",
  "dependencies": {
    "@aws-sdk/client-dynamodb": "3.172.0",
    "@aws-sdk/client-s3": "3.58.0"
  },
  "devDependencies": {
    "@types/node": "14.18.27",
    "npm": "8.3.0",
    "typescript": "4.6.4"
  },
  "overrides": {
    "@aws-sdk/types": "3.171.0"
  }
}

src/main.ts

import { DynamoDBClient } from "@aws-sdk/client-dynamodb"; // @aws-sdk/types 3.171.0
import { S3Client } from "@aws-sdk/client-s3"; // @aws-sdk/types 3.171.0
import { Client, MetadataBearer } from "@aws-sdk/types"; // @aws-sdk/types 3.171.0

const c1: Client<object, MetadataBearer, unknown> = new DynamoDBClient({});
const c2: Client<object, MetadataBearer, unknown> = new S3Client({});

marcogrcr commented 11 months ago

Update: there's a more sophisticated workaround. It turns out the auto-generated package.json overrides the hardcoded version numbers with the following file if present in the classpath: software/amazon/smithy/aws/typescript/codegen/sdkVersions.properties.

Actually, this file is auto-generated and included in software.amazon.smithy.typescript:smithy-aws-typescript-codegen. For example, version 0.12.0 has the following contents:

@aws-sdk/middleware-flexible-checksums=3.171.0
@aws-sdk/middleware-sdk-rds=3.171.0
@aws-sdk/util-utf8-browser=3.170.0
# (...)

You can override this file with your own, as long as the path to a folder or .jar that contains it appears in the class path before smithy-aws-typescript-codegen-{version}.jar. That being said, be careful before using ^3.0.0 everywhere for all @aws-sdk/* dependencies. This is because the auto-generated code may not be compatible with newer versions. For example, the following sdkVersions.properties is compatible with version smithy-typescript@0.17.1:

# Overrides the sdkVersions.properties file auto-generated by the JavaScript AWS SDK v3
# See: https://github.com/aws/aws-sdk-js-v3/blob/v3.439.0/codegen/smithy-aws-typescript-codegen/build.gradle.kts#L49-L68

# Version 3.377.0 is the last version that uses compatible `@smithy/*` dependencies.
# Version 3.378.0 upgraded `@smithy/*` dependencies to ^2.x.x
# See: https://github.com/aws/aws-sdk-js-v3/compare/v3.377.0...v3.378.0
# See: https://github.com/aws/aws-sdk-js-v3/commit/96938415e65ccc4353be8f816e919215de12a1b7

# NPM dependencies defined in `software.amazon.smithy.typescript:smithy-aws-typescript-codegen:0.17.1`
# See: https://github.com/aws/aws-sdk-js-v3/blob/86691ebf037e6919e335c5347c5ea1a4a1233f00/codegen/smithy-aws-typescript-codegen/src/main/java/software/amazon/smithy/aws/typescript/codegen/AwsDependency.java
# @aws-crypto/sha1-browser                      = 3.0.0  (hardcoded)
# @types/uuid                                   = ^8.3.0 (hardcoded)
# fast-xml-parser                               = 4.2.5  (hardcoded)
# uuid                                          = ^8.3.2 (hardcoded)
@aws-sdk/body-checksum-browser                  = 3.0.0 - 3.377.0
@aws-sdk/body-checksum-node                     = 3.0.0 - 3.377.0
@aws-sdk/client-sts                             = 3.0.0 - 3.377.0
@aws-sdk/credential-provider-node               = 3.0.0 - 3.377.0
@aws-sdk/eventstream-handler-node               = 3.0.0 - 3.377.0
@aws-sdk/hash-blob-browser                      = 3.0.0 - 3.377.0
@aws-sdk/hash-stream-node                       = 3.0.0 - 3.377.0
@aws-sdk/md5-js                                 = 3.0.0 - 3.377.0
@aws-sdk/middleware-api-key                     = 3.0.0 - 3.377.0
@aws-sdk/middleware-bucket-endpoint             = 3.0.0 - 3.377.0
@aws-sdk/middleware-endpoint-discovery          = 3.0.0 - 3.377.0
@aws-sdk/middleware-eventstream                 = 3.0.0 - 3.377.0
@aws-sdk/middleware-expect-continue             = 3.0.0 - 3.377.0
@aws-sdk/middleware-flexible-checksums          = 3.0.0 - 3.377.0
@aws-sdk/middleware-host-header                 = 3.0.0 - 3.377.0
@aws-sdk/middleware-location-constraint         = 3.0.0 - 3.377.0
@aws-sdk/middleware-logger                      = 3.0.0 - 3.377.0
@aws-sdk/middleware-recursion-detection         = 3.0.0 - 3.377.0
@aws-sdk/middleware-sdk-api-gateway             = 3.0.0 - 3.377.0
@aws-sdk/middleware-sdk-ec2                     = 3.0.0 - 3.377.0
@aws-sdk/middleware-sdk-eventbridge             = 3.0.0 - 3.377.0
@aws-sdk/middleware-sdk-glacier                 = 3.0.0 - 3.377.0
@aws-sdk/middleware-sdk-machinelearning         = 3.0.0 - 3.377.0
@aws-sdk/middleware-sdk-rds                     = 3.0.0 - 3.377.0
@aws-sdk/middleware-sdk-route53                 = 3.0.0 - 3.377.0
@aws-sdk/middleware-sdk-s3                      = 3.0.0 - 3.377.0
@aws-sdk/middleware-sdk-s3-control              = 3.0.0 - 3.377.0
@aws-sdk/middleware-sdk-sqs                     = 3.0.0 - 3.377.0
@aws-sdk/middleware-sdk-sts                     = 3.0.0 - 3.377.0
@aws-sdk/middleware-sdk-transcribe-streaming    = 3.0.0 - 3.377.0
@aws-sdk/middleware-signing                     = 3.0.0 - 3.377.0
@aws-sdk/middleware-ssec                        = 3.0.0 - 3.377.0
@aws-sdk/middleware-token                       = 3.0.0 - 3.377.0
@aws-sdk/middleware-user-agent                  = 3.0.0 - 3.377.0
@aws-sdk/middleware-websocket                   = 3.0.0 - 3.377.0
@aws-sdk/node-config-provider                   = 3.0.0 - 3.377.0
@aws-sdk/signature-v4-multi-region              = 3.0.0 - 3.377.0
@aws-sdk/util-user-agent-browser                = 3.0.0 - 3.377.0
@aws-sdk/util-user-agent-node                   = 3.0.0 - 3.377.0
@aws-sdk/xml-builder                            = 3.0.0 - 3.377.0

# NPM dependencies defined in `software.amazon.smithy.typescript:smithy-typescript-codegen:0.17.1`
# See: https://github.com/awslabs/smithy-typescript/blob/7cf055fc62527622ac52b5696d84bea0f1e54af8/smithy-typescript-codegen/src/main/java/software/amazon/smithy/typescript/codegen/TypeScriptDependency.java
# @aws-crypto/sha256-browser                     = 3.0.0          (harcoded)
# @aws-crypto/sha256-js                          = 3.0.0          (hardcoded)
# @aws-smithy/server-common                      = 1.0.0-alpha.10 (hardcoded)
# @smithy/config-resolver                        = ^1.0.1         (hardcoded)
# @smithy/eventstream-serde-browser              = ^1.0.1         (hardcoded)
# @smithy/eventstream-serde-config-resolver      = ^1.0.1         (hardcoded)
# @smithy/eventstream-serde-node                 = ^1.0.1         (hardcoded)
# @smithy/fetch-http-handler                     = ^1.0.1         (hardcoded)
# @smithy/hash-blob-browser                      = ^1.0.1         (hardcoded)
# @smithy/hash-node                              = ^1.0.1         (hardcoded)
# @smithy/hash-stream-node                       = ^1.0.1         (hardcoded)
# @smithy/invalid-dependency                     = ^1.0.1         (hardcoded)
# @smithy/md5-js                                 = ^1.0.1         (hardcoded)
# @smithy/middleware-apply-body-checksum         = ^1.0.1         (hardcoded)
# @smithy/middleware-content-length              = ^1.0.1         (hardcoded)
# @smithy/middleware-endpoint                    = ^1.0.2         (hardcoded)
# @smithy/middleware-retry                       = ^1.0.3         (hardcoded)
# @smithy/middleware-serde                       = ^1.0.1         (hardcoded)
# @smithy/middleware-stack                       = ^1.0.1         (hardcoded)
# @smithy/node-config-provider                   = ^1.0.1         (hardcoded)
# @smithy/node-http-handler                      = ^1.0.2         (hardcoded)
# @smithy/protocol-http                          = ^1.1.0         (hardcoded)
# @smithy/querystring-builder                    = ^1.0.1         (hardcoded)
# @smithy/service-client-documentation-generator = ^1.0.1         (hardcoded)
# @smithy/smithy-client                          = ^1.0.3         (hardcoded)
# @smithy/types                                  = ^1.1.0         (hardcoded)
# @smithy/url-parser                             = ^1.0.1         (hardcoded)
# @smithy/util-base64                            = ^1.0.1         (hardcoded)
# @smithy/util-base64-browser                    = ^1.0.1         (hardcoded)
# @smithy/util-base64-node                       = ^1.0.1         (hardcoded)
# @smithy/util-body-length-browser               = ^1.0.1         (hardcoded)
# @smithy/util-body-length-node                  = ^1.0.1         (hardcoded)
# @smithy/util-defaults-mode-browser             = ^1.0.1         (hardcoded)
# @smithy/util-defaults-mode-node                = ^1.0.1         (hardcoded)
# @smithy/util-middleware                        = ^1.0.1         (hardcoded)
# @smithy/util-retry                             = ^1.0.3         (hardcoded)
# @smithy/util-stream                            = ^1.0.1         (hardcoded)
# @smithy/util-stream-browser                    = ^1.0.1         (hardcoded)
# @smithy/util-stream-node                       = ^1.0.1         (hardcoded)
# @smithy/util-utf8                              = ^1.0.1         (hardcoded)
# @smithy/util-waiter                            = ^1.0.1         (hardcoded)
# @types/big.js                                  = ^6.0.0         (hardcoded)
# @types/node                                    = ^14.14.31      (hardcoded)
# big.js                                         = ^6.0.0         (hardcoded)
# entities                                       = 2.2.0          (hardcoded)
# fast-xml-parser                                = 4.2.5          (hardcoded)
# uuid                                           = ^8.3.2         (hardcoded)
@aws-sdk/types                                   = 3.0.0 - 3.377.0
@aws-sdk/util-endpoints                          = 3.0.0 - 3.377.0

smithy-lang / smithy-typescript

Use ^3.0.0 versions of client-generated dependencies #602