search

aws / aws-sdk-js-v3

Modularized AWS SDK for JavaScript.
Apache License 2.0
3.12k stars 579 forks source link

Error Module has no exported member 'UserAgent' while using aws-sdk-js-v3 #1842

Closed vishwasrao closed 3 years ago

vishwasrao commented 3 years ago

Describe the bug I am using aws-sdk-js-v3 in my typescript project, I installed package with command yarn add @aws-sdk/client-s3. After using S3Client in my typescript code 

import { S3Client } from '@aws-sdk/client-s3';

Application starts failing with error

node_modules/@aws-sdk/client-s3/types/S3Client.d.ts:102:392 - error TS2305: Module '"../../types/dist/cjs"' has no exported member 'UserAgent'.
102 import { Provider, RegionInfoProvider, Credentials as __Credentials, Decoder as __Decoder, Encoder as __Encoder, EventStreamSerdeProvider as __EventStreamSerdeProvider, HashConstructor as __HashConstructor, HttpHandlerOptions as __HttpHandlerOptions, Logger as __Logger, Provider as __Provider, StreamCollector as __StreamCollector, StreamHasher as __StreamHasher, UrlParser as __UrlParser, UserAgent as __UserAgent } from "@aws-sdk/types";  
                                                                                   ~~~~~~~~~
node_modules/@aws-sdk/middleware-user-agent/dist/cjs/configurations.d.ts:1:20 - error TS2305: Module '"../../../types/dist/cjs"' has no exported member 'UserAgent'.

1 import { Provider, UserAgent } from "@aws-sdk/types";

SDK version number 3.1.0

Is the issue in the browser/Node.js/ReactNative? Node.js

Details of the Node.js version $ node -v v12.4.0

To Reproduce (observed behavior)

  1. Install package with command yarn add @aws-sdk/client-s3
  2. Import S3Client in typescript file import { S3Client } from '@aws-sdk/client-s3'; and use client to get object from S3.
  3. Start typescript application with command yarn run start:Dev
  4. It fails to start with given command

Note: If I install package @aws-sdk/types with command yarn add @aws-sdk/types then this error goes away and works as expected. Do we really need @aws-sdk/types package, Expectation was @aws-sdk/client-s3 will be sufficient.

Expected behavior It should not give any error and extra package @aws-sdk/types should not be needed

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here.

mfogel commented 3 years ago

I am running into this error as well. I'm using v3.1.0 of the SDK.

I see it with a few other clients as well

What appears to be going is that all three of these clients have a dependencies on

I'm using yarn and when I grep through my node_modules, I don't see @aws-sdk/types@3.1.0 only @aws-sdk/types@^1.0.0-rc.1. Hence when these clients are trying to import from @aws-sdk/types they're getting v1.0.0-rc1 when they are expecting to get v3.1.0

It looks like this was probably introduced in https://github.com/aws/aws-sdk-js-v3/pull/1775

It looks like this would be fixed by https://github.com/aws/aws-sdk-js-crypto-helpers/pull/187

Adding this to my package.json is working as a temporary work-around for me (again, using yarn)

"resolutions": {
  "@aws-sdk/types": "^3.1.0"
}
kirintwn commented 3 years ago

Using the the above temporary work-around resolutions in package.json is not working for me (using npm).

I have to explicitly install @aws-sdk/types as one of devDependencies, then the problems are gone:

npm install -D @aws-sdk/types@3.1.0
jamesjrg commented 3 years ago

Installing @aws-sdk/types as above made this error go away, but then I got another error:

node_modules/@aws-sdk/types/dist/cjs/util.d.ts:92:42 - error TS1005: ',' expected.

92 export declare type UserAgentPair = [name: string, version?: string];
                                            ~
node_modules/@aws-sdk/types/dist/cjs/util.d.ts:92:60 - error TS1005: ',' expected.

92 export declare type UserAgentPair = [name: string, version?: string];

I guess that package doesn't work with strict mode enabled.

prime417 commented 3 years ago

Installing @aws-sdk/types as above made this error go away, but then I got another error:

node_modules/@aws-sdk/types/dist/cjs/util.d.ts:92:42 - error TS1005: ',' expected.

92 export declare type UserAgentPair = [name: string, version?: string];
                                            ~
node_modules/@aws-sdk/types/dist/cjs/util.d.ts:92:60 - error TS1005: ',' expected.

92 export declare type UserAgentPair = [name: string, version?: string];

I guess that package doesn't work with strict mode enabled.

I am also getting same issue.Looking for any solution.

olokobayusuf commented 3 years ago

Hey everyone, I just wanted to let you know that the following seems to fix this issue:

  1. As @kirintwn mentioned, install the types package: 
    npm install -D @aws-sdk/types@3.1.0
  2. Then upgrade Typescript, I'm on 4.1.3: 
    npm install typescript@latest

    I hope this is useful.

peterlogo commented 3 years ago

@olokobayusuf thanks your solution worked for me.

sozakir commented 3 years ago

I am also getting the same error as @prime417 after explicitly installing types.

As I am using angular 9.1.12 which uses typescript version 3.8.3, as soon as I switched to 4.1.3, I am getting this error. In order to have this hack work for me I need to upgrade the angular version to 10.1.0 or above.

Compiling TypeScript sources through ngc
ERROR: The Angular Compiler requires TypeScript >=3.6.4 and <3.9.0 but 4.1.3 was found instead.
An unhandled exception occurred: The Angular Compiler requires TypeScript >=3.6.4 and <3.9.0 but 4.1.3 was found instead.

waiting for it to be resolved and any other suggestions?

BCarroll524 commented 3 years ago

@olokobayusuf your solution has also worked for me. Thank you

AllanZhengYP commented 3 years ago

Thanks everyone for the thorough investigation. @olokobayusuf 's solution should solve the problem. Although the @aws-sdk/client-s3@3.1.0 should have pinned dependency over @aws-sdk/types@3.1.0. We need to dive deeper into this on our end.

olokobayusuf commented 3 years ago

Thanks everyone for the thorough investigation. @olokobayusuf 's solution should solve the problem. Although the @aws-sdk/client-s3@3.1.0 should have pinned dependency over @aws-sdk/types@3.1.0. We need to dive deeper into this on our end.

Hey there @AllanZhengYP I hate to discuss a different issue, but my development is completely blocked on #1857 . I don't mind making a custom fork of the SDK with a fix while you all work on an official update, but I need a fix ASAP.

p-obrien commented 3 years ago

This isn't working for me with the EC2 Client, after applying @olokobayusuf solution I'm getting the following:

node_modules/@aws-sdk/types/dist/cjs/util.d.ts:92:42` - error TS1005: ',' expected.
92 export declare type UserAgentPair = [name: string, version?: string];

node_modules/@aws-sdk/types/dist/cjs/util.d.ts:92:60 - error TS1005: ',' expected.
92 export declare type UserAgentPair = [name: string, version?: string];

Thanks everyone for the thorough investigation. @olokobayusuf 's solution should solve the problem. Although the @aws-sdk/client-s3@3.1.0 should have pinned dependency over @aws-sdk/types@3.1.0. We need to dive deeper into this on our end.

trivikr commented 3 years ago

I don't mind making a custom fork of the SDK with a fix while you all work on an official update, but I need a fix ASAP.

@olokobayusuf You can also post a PR with a fix on this repo.

olokobayusuf commented 3 years ago

This isn't working for me with the EC2 Client, after applying @olokobayusuf solution I'm getting the following:

node_modules/@aws-sdk/types/dist/cjs/util.d.ts:92:42` - error TS1005: ',' expected.
92 export declare type UserAgentPair = [name: string, version?: string];

node_modules/@aws-sdk/types/dist/cjs/util.d.ts:92:60 - error TS1005: ',' expected.
92 export declare type UserAgentPair = [name: string, version?: string];

Thanks everyone for the thorough investigation. @olokobayusuf 's solution should solve the problem. Although the @aws-sdk/client-s3@3.1.0 should have pinned dependency over @aws-sdk/types@3.1.0. We need to dive deeper into this on our end.

Check your Typescript version. You're likely getting this error because you're running on TS 3.x.

olokobayusuf commented 3 years ago

I don't mind making a custom fork of the SDK with a fix while you all work on an official update, but I need a fix ASAP.

@olokobayusuf You can also post a PR with a fix on this repo.

That's fine by me, I just need to know how to fix it. I haven't done too much static analysis on the codebase because I haven't had the time. I'm unfamiliar with the middleware stack, but I'm guessing any fix would be either working around the middleware stack, or making the keys within the stack unique so that the conflict never pops up (this would be harder because there isn't a global state for knowing how many requests are in flight AFAIK, so I'd have to resort to something hacky like appending a UUID to the key for each entry). I just need a quick rundown and I'll try to hash it out as quickly as possible.

trivikr commented 3 years ago

This issue is happening as @aws-sdk/types is declared as a dependency instead of devDependency in the following packages:

$ yarn why @aws-sdk/types
yarn why v1.22.10
warning package.json: No license field
[1/4] 🤔  Why do we have the module "@aws-sdk/types"...?
[2/4] 🚚  Initialising dependency graph...
warning No license field
[3/4] 🔍  Finding dependency...
[4/4] 🚡  Calculating file sizes...
=> Found "@aws-sdk/types@1.0.0-rc.3"
info Has been hoisted to "@aws-sdk/types"
info Reasons this module exists
   - Hoisted from "@aws-sdk#client-s3#@aws-crypto#sha256-browser#@aws-sdk#types"
   - Hoisted from "@aws-sdk#client-s3#@aws-crypto#sha256-js#@aws-sdk#types"
info Disk size without dependencies: "524KB"
info Disk size with unique dependencies: "524KB"
info Disk size with transitive dependencies: "524KB"
info Number of shared dependencies: 0
=> Found "@aws-sdk/util-user-agent-browser#@aws-sdk/types@3.1.0"
info This module exists because "@aws-sdk#client-s3#@aws-sdk#util-user-agent-browser" depends on it.
info Disk size without dependencies: "540KB"
info Disk size with unique dependencies: "540KB"
info Disk size with transitive dependencies: "540KB"
info Number of shared dependencies: 0
=> Found "@aws-sdk/util-user-agent-node#@aws-sdk/types@3.1.0"
info This module exists because "@aws-sdk#client-s3#@aws-sdk#util-user-agent-node" depends on it.
info Disk size without dependencies: "540KB"
info Disk size with unique dependencies: "540KB"
info Disk size with transitive dependencies: "540KB"
info Number of shared dependencies: 0
✨  Done in 0.16s.

ToDo: Move @aws-sdk/types to devDependency in the affected packages.

trivikr commented 3 years ago

Relevant PRs:

ykoglin commented 3 years ago

This isn't working for me with the EC2 Client, after applying @olokobayusuf solution I'm getting the following:

node_modules/@aws-sdk/types/dist/cjs/util.d.ts:92:42` - error TS1005: ',' expected.
92 export declare type UserAgentPair = [name: string, version?: string];

node_modules/@aws-sdk/types/dist/cjs/util.d.ts:92:60 - error TS1005: ',' expected.
92 export declare type UserAgentPair = [name: string, version?: string];

Thanks everyone for the thorough investigation. @olokobayusuf 's solution should solve the problem. Although the @aws-sdk/client-s3@3.1.0 should have pinned dependency over @aws-sdk/types@3.1.0. We need to dive deeper into this on our end.

Check your Typescript version. You're likely getting this error because you're running on TS 3.x. tsc -v Version 4.1.3 "dependencies": "@aws-sdk/client-dynamodb": "^3.1.0", "@aws-sdk/client-s3": "^3.1.0", "@aws-sdk/util-dynamodb": "^3.1.0", "devDependencies": "@aws-sdk/types": "^3.1.0",

node_modules/@aws-sdk/types/dist/cjs/util.d.ts:92:42 - error TS1005: ',' expected.

92 export declare type UserAgentPair = [name: string, version?: string]; ~

node_modules/@aws-sdk/types/dist/cjs/util.d.ts:92:60 - error TS1005: ',' expected.

92 export declare type UserAgentPair = [name: string, version?: string];

mfogel commented 3 years ago

Still seeing this bug with yesterday's release of the SDK, v3.2.0, on both typescript 4.0 & 4.1.

Since a couple PR's were merged in over in https://github.com/aws/aws-sdk-js-crypto-helpers, I think this is now just blocked on a new release of the various packages from that repo.

/cc @seebees

trivikr commented 3 years ago

Since a couple PR's were merged in over in https://github.com/aws/aws-sdk-js-crypto-helpers, I think this is now just blocked on a new release of the various packages from that repo.

The next version of crypto-helpers is expected to release on Wed 1/13. We'll release JS SDK v3 v3.3.0 consuming new crypto-helpers version by Thu 1/14

alexforsyth commented 3 years ago

Posting this here for documentation and visibility.

TLDR: We're moving @aws-sdk/types to dependencies so that you get types right out of the box at the cost of an increase of 492kb in package size. This will not impact users that use bundlers.

After some investigation we've come to the conclusion that @aws-sdk/types should be exposed as a "dependency" in all packages where a non-test file depends on types. I'll be making updates to @aws-crypto/* and @aws-sdk/* shortly. I've published my own versions to my local npm registry to confirm that these updates should fix this family of issues moving forward.

Background Customers consume @aws-sdk/* with types but we declare @aws-sdk/types as a devDependency in all packages in this repo. @trivikr showed in https://github.com/aws/aws-sdk-js-v3/issues/1842#issuecomment-755738925 that right now customers are getting access to @aws-sdk/types somewhat erroneously through a version of @aws-crypto/sha256-{js, browser} which brings in the most recent rc version of @aws-sdk/types as a dependency, since this is the only place where @aws-sdk/types are declared as a dependency, this is what consumers of @aws-sdk are actually using for @aws-sdk/types under the hood and why it hadn't broken until types were updated with useragent in 3.1.0.

Like @olokobayusuf mentioned including @aws-sdk/types as a dependency yourself will also fix the issue.

Why are we doing this? When a package is declared as a devDependency it is (rightfully) not propagated to downstream consumers of that package. We publish packages that are consumed by downstream users, they should have access to types right out of the box without the need to install @aws-sdk/types themselves. Since we want our consumers to automatically get our type declarations we must declare these types as dependencies. See: https://github.com/Microsoft/types-publisher/issues/81#issuecomment-234051345

Wont this increase node module size? Yes, it will increase node module size by ~492kb. This is an issue brought up https://github.com/aws/aws-sdk-js-v3/issues/1649. However, @DanielRosenwasser brings up a good point in https://github.com/Microsoft/types-publisher/issues/81#issuecomment-234051345 that

breaking consumers is a worse problem than slightly-larger packages, we've made --save the default in our documentation.

The AWS SDK team (and I personally) agree. Additionally most JS users use bundlers anyway that should strip out these files.

alexforsyth commented 3 years ago

This should be going out with our next release. Huge shout out to @olokobayusuf!

MarrickLip commented 3 years ago

@alexforsyth glad to see this fixed!

Is it worth reviewing how this got through testing? This feels to me like the sort of thing that would be caught in CICD etc. before ever shipping.

alexforsyth commented 3 years ago

@MarrickLip We're working on it, I dont have anything I can post to the public quite yet

markusl commented 3 years ago

@alexforsyth just to confirm, the new release is also supposed to fix the already reported issue about this

node_modules/@aws-sdk/types/dist/cjs/util.d.ts:92:42` - error TS1005: ',' expected.
92 export declare type UserAgentPair = [name: string, version?: string];

Just checking if it needs another issue to be fixed. I'm currently hitting this when using explicit dependency to @aws-sdk/types.

alexforsyth commented 3 years ago

@markusl yes it should be fixed. If you run yarn why @aws-sdk/types or npm ls @aws-sdk/types in your repo currently it should show something like this:

yarn why @aws-sdk/types
yarn why v1.22.5
[1/4] 🤔  Why do we have the module "@aws-sdk/types"...?
[2/4] 🚚  Initialising dependency graph...
[3/4] 🔍  Finding dependency...
[4/4] 🚡  Calculating file sizes...
=> Found "@aws-sdk/types@1.0.0-rc.10"
info Reasons this module exists
   - "@aws-crypto#sha256-js" depends on it
   - Hoisted from "@aws-crypto#sha256-js#@aws-sdk#types"
   - Hoisted from "@aws-sdk#client-dynamodb#@aws-crypto#sha256-browser#@aws-sdk#types"
info Disk size without dependencies: "528KB"
info Disk size with unique dependencies: "528KB"
info Disk size with transitive dependencies: "528KB"
info Number of shared dependencies: 0

In the next version there will be no dependencies on "@aws-sdk/types@1.0.0-rc.10" so you should be getting the correct types. I believe you can still mitigate this issue with the current release by taking a dependency on @aws-sdk/types@^3.1.0 yourself. I actually just ran this myself to confirm. If you're still encountering an issue after you've taken the @aws-sdk/types dependency yourself, can you post the results of your yarn why?

mfogel commented 3 years ago

As of yesterday's v3.3.0 release, this issue appears to be fixed. No more need for the workarounds. Thanks for getting that release out last night AWS team.

adaadamczak007 commented 3 years ago

Hi, still having issues... is v3.3.0 supposed to work with typescript 3.7.x?

I still see after upgrade to v3.3.0:

node_modules/@aws-sdk/types/dist/cjs/util.d.ts:92:60 - error TS1005: ',' expected.

92 export declare type UserAgentPair = [name: string, version?: string];
alexforsyth commented 3 years ago

@adaadamczak007 please upgrade to typescript ^4.0.0. This issue is slightly different and tracked here: https://github.com/aws/aws-sdk-js-v3/issues/1919.

TS doesn’t have LTS concept, they drop support for previous version when the new one is released. I'll post updates when I can make more public, we're working on guidance around this internally!

alexforsyth commented 3 years ago

Closing this bug as 3.3.x should solve this family of issues! Thanks for your support all!

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.