search

aws-amplify / amplify-js

A declarative JavaScript library for application development using cloud services.
https://docs.amplify.aws/lib/q/platform/js
Apache License 2.0
9.42k stars 2.12k forks source link

Storage ExpiredToken: The provided token has expired #12787

Open kolodi opened 8 months ago

kolodi commented 8 months ago

Before opening, please confirm:

JavaScript Framework

Next.js

Amplify APIs

Authentication, Storage

Amplify Categories

auth, storage

Environment information

``` # Put output below this line Packages System: OS: Linux 5.15 Ubuntu 20.04.4 LTS (Focal Fossa) CPU: (12) x64 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz Memory: 1.67 GB / 7.65 GB Container: Yes Shell: 5.0.17 - /bin/bash Binaries: Node: 18.16.0 - ~/.nvm/versions/node/v18.16.0/bin/node Yarn: 1.22.19 - ~/.nvm/versions/node/v18.16.0/bin/yarn npm: 9.7.1 - ~/.nvm/versions/node/v18.16.0/bin/npm npmPackages: @ampproject/toolbox-optimizer: undefined () @aws-amplify/ui-react: ^5.0.2 => 5.0.2 @aws-amplify/ui-react-internal: undefined () @babel/core: undefined () @babel/runtime: 7.15.4 @edge-runtime/cookies: 3.4.1 @edge-runtime/ponyfill: 2.4.0 @edge-runtime/primitives: 3.1.1 @hapi/accept: undefined () @mswjs/interceptors: undefined () @napi-rs/triples: undefined () @next/font: undefined () @next/react-dev-overlay: undefined () @opentelemetry/api: undefined () @react-three/drei: ^9.88.4 => 9.88.4 @react-three/fiber: ^8.15.4 => 8.15.4 @react-three/postprocessing: ^2.15.1 => 2.15.1 @segment/ajv-human-errors: undefined () @types/debounce: ^1.2.1 => 1.2.1 @types/deep-equal: ^1.0.1 => 1.0.1 @types/mime-types: ^2.1.1 => 2.1.1 @types/node: ^18.16.18 => 18.16.18 @types/react: ^18.2.13 => 18.2.13 @types/react-copy-to-clipboard: ^5.0.4 => 5.0.4 @types/react-currency-format: ^1.0.0 => 1.0.0 @types/react-dom: ^18.2.6 => 18.2.6 @types/react-modal: ^3.16.0 => 3.16.0 @types/react-resizable: ^3.0.4 => 3.0.4 @types/three: ^0.152.1 => 0.152.1 @vercel/nft: undefined () @vercel/og: undefined () acorn: undefined () amphtml-validator: undefined () anser: undefined () arg: undefined () assert: undefined () async-retry: undefined () async-sema: undefined () autoprefixer: ^10.4.14 => 10.4.14 aws-amplify: ^5.3.1 => 5.3.12 aws-sdk: ^2.1403.0 => 2.1403.0 babel-packages: undefined () browserify-zlib: undefined () browserslist: undefined () buffer: undefined () bytes: undefined () ci-info: undefined () cli-select: undefined () client-only: 0.0.1 clsx: ^2.0.0 => 2.0.0 (1.2.1) comment-json: undefined () compression: undefined () conf: undefined () constants-browserify: undefined () content-disposition: undefined () content-type: undefined () cookie: undefined () cross-spawn: undefined () crypto-browserify: undefined () css.escape: undefined () data-uri-to-buffer: undefined () debounce: ^1.2.1 => 1.2.1 debug: undefined () deep-equal: ^2.2.1 => 2.2.1 devalue: undefined () domain-browser: undefined () dotenv: ^16.3.1 => 16.3.1 edge-runtime: undefined () eslint: ^8.43.0 => 8.43.0 eslint-config-next: ^13.4.7 => 13.4.7 events: undefined () find-cache-dir: undefined () find-up: undefined () fresh: undefined () get-orientation: undefined () glob: ^10.3.0 => undefined (7.1.7, 7.2.3, 10.3.0, , 7.1.6) gzip-size: undefined () http-proxy: undefined () http-proxy-agent: undefined () https-browserify: undefined () https-proxy-agent: undefined () icss-utils: undefined () ignore-loader: undefined () image-size: undefined () is-animated: undefined () is-docker: undefined () is-wsl: undefined () jest-worker: undefined () json5: undefined () jsonwebtoken: undefined () leva: ^0.9.35 => 0.9.35 loader-runner: undefined () loader-utils: undefined () local-ssl-proxy: ^2.0.5 => 2.0.5 lodash.curry: undefined () lru-cache: undefined () micromatch: undefined () mime-types: ^2.1.35 => 2.1.35 mini-css-extract-plugin: undefined () nanoid: undefined () native-url: undefined () neo-async: undefined () next: ^13.4.7 => 13.5.4 next-auth: ^4.22.1 => 4.24.5 node-fetch: undefined () node-html-parser: undefined () ora: undefined () os-browserify: undefined () p-limit: undefined () path-browserify: undefined () platform: undefined () postcss: ^8.4.24 => 8.4.31 postcss-flexbugs-fixes: undefined () postcss-modules-extract-imports: undefined () postcss-modules-local-by-default: undefined () postcss-modules-scope: undefined () postcss-modules-values: undefined () postcss-preset-env: undefined () postcss-safe-parser: undefined () postcss-scss: undefined () postcss-value-parser: undefined () process: undefined () punycode: undefined () qrcode.react: ^3.1.0 => 3.1.0 querystring-es3: undefined () raw-body: undefined () react: ^18.2.0 => 18.2.0 react-arborist: ^3.2.0 => 3.2.0 react-builtin: undefined () react-copy-to-clipboard: ^5.1.0 => 5.1.0 react-device-detect: ^2.2.3 => 2.2.3 react-dom: ^18.2.0 => 18.2.0 react-dom-builtin: undefined () react-dom-experimental-builtin: undefined () react-experimental-builtin: undefined () react-hotkeys-hook: ^4.4.1 => 4.4.1 react-icons: ^4.10.1 => 4.10.1 react-is: 18.2.0 react-modal: ^3.16.1 => 3.16.1 react-refresh: 0.12.0 react-resizable: ^3.0.5 => 3.0.5 react-server-dom-webpack-builtin: undefined () react-server-dom-webpack-experimental-builtin: undefined () regenerator-runtime: 0.13.4 sass-loader: undefined () scheduler-builtin: undefined () scheduler-experimental-builtin: undefined () schema-utils: undefined () semver: undefined () send: undefined () server-only: 0.0.1 setimmediate: undefined () shell-quote: undefined () source-map: undefined () stacktrace-parser: undefined () stream-browserify: undefined () stream-http: undefined () string-hash: undefined () string_decoder: undefined () strip-ansi: undefined () superstruct: undefined () tailwindcss: ^3.3.2 => 3.3.2 tar: undefined () terser: undefined () text-table: undefined () three: ^0.157.0 => 0.157.0 timers-browserify: undefined () tty-browserify: undefined () typescript: ^5.1.3 => 5.1.3 ua-parser-js: undefined () undici: undefined () unistore: undefined () use-resize-observer: ^9.1.0 => 9.1.0 usehooks-ts: ^2.9.1 => 2.9.1 util: undefined () vm-browserify: undefined () watchpack: undefined () web-vitals: undefined () webpack: undefined () webpack-sources: undefined () ws: undefined () zustand: ^4.3.8 => 4.3.8 (3.7.2) npmGlobalPackages: @aws-amplify/cli: 12.1.1 corepack: 0.18.1 npm-check-updates: 16.10.12 npm: 9.7.1 ts-node: 10.9.1 ```

Describe the bug

Storage operations fail due to token expiration. Amplify should take care of refreshing tokens automatically but it is not working for Storage for some reason. I still can do other operations like calling GraphQL and REST APIs, but the Storage is working only when refreshing completely the page.

Expected behavior

Storage.put ot Storage.get should be using refreshed tokens as other services do.

Reproduction steps

  1. Login with standard Authentication from (for React)
  2. Navigate to pages and use different APIs including Storage (browser uploads)
  3. After some time (30m - 1hr) the Storage put fail due to token expiration

Code Snippet

import { Storage } from "aws-amplify";

export async function uploadFileToS3Simple(file: File | Blob, fileName: string) {
    const result = await Storage.put(fileName, file, {
        level: "public",
    });
}

Log output

``` // Put your logs below this line [DEBUG] 12:22.941 Credentials - getting credentials s3.ts:23 [DEBUG] 12:22.941 Credentials - picking up credentials s3.ts:23 [DEBUG] 12:22.941 Credentials - getting new cred promise s3.ts:23 [DEBUG] 12:22.942 Credentials - checking if credentials exists and not expired s3.ts:23 [DEBUG] 12:22.942 Credentials - are these credentials expired? {accessKeyId: 'ASIA2DQVNAKBJT3YOW5T', secretAccessKey: 'sSUyphelVjErIy1QppdFalzlx5UojCjSBl3Obn1i', sessionToken: 'IQoJb3JpZ2luX2VjEMj//////////wEaCWV1LXdlc3QtMSJIME…XsLQE90eBAKqy65PoJPnlZdfuFfKz5em6NWskgjw+XIhDpjE9', expiration: Wed Jan 03 2024 17:48:26 GMT+0100 (Central European Standard Time), identityId: 'eu-west-1:2ca45818-e464-4703-9c0e-3ae2d3d9012d', …} s3.ts:23 [DEBUG] 12:22.942 Credentials - credentials not changed and not expired, directly return ConsoleLogger.js:134 [DEBUG] 12:22.942 S3ClientUtils - credentials provider get credentials {accessKeyId: 'ASIA2DQVNAKBJT3YOW5T', sessionToken: 'IQoJb3JpZ2luX2VjEMj//////////wEaCWV1LXdlc3QtMSJIME…XsLQE90eBAKqy65PoJPnlZdfuFfKz5em6NWskgjw+XIhDpjE9', secretAccessKey: 'sSUyphelVjErIy1QppdFalzlx5UojCjSBl3Obn1i', identityId: 'eu-west-1:2ca45818-e464-4703-9c0e-3ae2d3d9012d', authenticated: true} ConsoleLogger.js:126 [DEBUG] 12:23.287 xhr-http-handler ProgressEvent {isTrusted: true, lengthComputable: true, loaded: 2055352, total: 2055352, type: 'progress', …}isTrusted: truebubbles: falsecancelBubble: falsecancelable: falsecomposed: falsecurrentTarget: XMLHttpRequestUpload {onloadstart: null, onprogress: null, onabort: null, onerror: null, onload: null, …}defaultPrevented: falseeventPhase: 0lengthComputable: trueloaded: 2055352returnValue: truesrcElement: XMLHttpRequestUpload {onloadstart: null, onprogress: null, onabort: null, onerror: null, onload: null, …}target: XMLHttpRequestUpload {onloadstart: null, onprogress: null, onabort: null, onerror: null, onload: null, …}timeStamp: 20296676.89999999total: 2055352type: "progress"[[Prototype]]: ProgressEvent s3.ts:23 PUT https://k3eportal7595ba4427b4495589b0e8d05220f261201813-dev.s3.eu-west-1.amazonaws.com/public/f4ad8202-8da6-450f-a5cd-1080c180ea2f/variants/fe904e39-c526-4c66-a4fb-b96ee5b29c33.glb 400 (Bad Request) ConsoleLogger.js:126 [DEBUG] 12:23.595 xhr-http-handler ProgressEvent {isTrusted: true, lengthComputable: false, loaded: 1560, total: 0, type: 'progress', …} useGeometryEdit.tsx:124 ExpiredToken: The provided token has expired. at eval (webpack-internal:///./node_modules/@aws-amplify/storage/lib-esm/AwsClients/S3/utils/parsePayload.js:67:25) at step (webpack-internal:///./node_modules/@aws-amplify/storage/lib-esm/AwsClients/S3/utils/parsePayload.js:41:23) at Object.eval [as next] (webpack-internal:///./node_modules/@aws-amplify/storage/lib-esm/AwsClients/S3/utils/parsePayload.js:22:53) at fulfilled (webpack-internal:///./node_modules/@aws-amplify/storage/lib-esm/AwsClients/S3/utils/parsePayload.js:13:58) ```

aws-exports.js

const awsmobile = {
    "aws_project_region": "eu-west-1",
    "aws_cloud_logic_custom": [
        {
            "name": "AdminQueries",
            "endpoint": "https://8w0stnvaud.execute-api.eu-west-1.amazonaws.com/dev",
            "region": "eu-west-1"
        },
        {
            "name": "k3epub",
            "endpoint": "https://07x954irgj.execute-api.eu-west-1.amazonaws.com/dev",
            "region": "eu-west-1"
        }
    ],
    "aws_appsync_graphqlEndpoint": "https://arvu2uwwvneyhpaysrwujvygme.appsync-api.eu-west-1.amazonaws.com/graphql",
    "aws_appsync_region": "eu-west-1",
    "aws_appsync_authenticationType": "AMAZON_COGNITO_USER_POOLS",
    "aws_cognito_identity_pool_id": "eu-west-1:81e28009-f26b-45a0-8cb3-0b3c9128073d",
    "aws_cognito_region": "eu-west-1",
    "aws_user_pools_id": "eu-west-1_4ucIMRSmO",
    "aws_user_pools_web_client_id": "1qfnsdsi5l97ua0pf75b9ki6uj",
    "oauth": {},
    "aws_cognito_username_attributes": [
        "EMAIL"
    ],
    "aws_cognito_social_providers": [],
    "aws_cognito_signup_attributes": [
        "EMAIL"
    ],
    "aws_cognito_mfa_configuration": "OFF",
    "aws_cognito_mfa_types": [
        "SMS"
    ],
    "aws_cognito_password_protection_settings": {
        "passwordPolicyMinLength": 8,
        "passwordPolicyCharacters": []
    },
    "aws_cognito_verification_mechanisms": [
        "EMAIL"
    ],
    "aws_user_files_s3_bucket": "k3eportal7595ba4427b4495589b0e8d05220f261201813-dev",
    "aws_user_files_s3_bucket_region": "eu-west-1"
};

Manual configuration

No response

Additional configuration

Additional Storage configurations

{
  "resourceName": "k3e",
  "policyUUID": "f8341cd2",
  "bucketName": "k3eportal7595ba4427b4495589b0e8d05220f261",
  "storageAccess": "auth",
  "guestAccess": [],
  "authAccess": [
    "CREATE_AND_UPDATE",
    "READ",
    "DELETE"
  ],
  "groupAccess": {
    "Admins": [
      "CREATE_AND_UPDATE",
      "READ",
      "DELETE"
    ],
    "OrgAdmins": [
      "CREATE_AND_UPDATE",
      "READ",
      "DELETE"
    ]
  }
}

Mobile Device

No response

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

No response

nadetastic commented 8 months ago

Hi @kolodi thank you for opening this issue. I see that you are using an Next JS app, does this issue seem to be specific to Server Side pages? Or do you see it regardless of client side/server side usage?

Also I'm working through testing this myself but curious if you have had a chance to see this happen on a SPA app.

kolodi commented 8 months ago

Hi @kolodi thank you for opening this issue. I see that you are using an Next JS app, does this issue seem to be specific to Server Side pages? Or do you see it regardless of client side/server side usage?

Also I'm working through testing this myself but curious if you have had a chance to see this happen on a SPA app.

Hi @nadetastic, You are tight. I have older "vanilla" js/ts (webpack) SPA application. It is using the Storage in the same way as this new Next app, infact, it is using the same Amplify backend (it was a single module of the project that I am almost dome migrating under the common next js application). And the browser upload to S3 it is working fine there, even after 1 hour after login.

Btw, I have ssr=true in my Amplify config.

import config from '../src/aws-exports';

Amplify.configure({

    ...config,
    ssr: true,
});
kolodi commented 8 months ago

So it must be related to SSR in next.js. Any expert in this area can take a look?

nadetastic commented 7 months ago

@kolodi it looks like you are using an older major version of aws-amplify (v5.x.x) which had limited support for SSR. If so, I recommend upgrading to the latest major version (v6) which as improved support for SSR. Here are some reference documentation on how you can upgrade and use SSR:

kolodi commented 7 months ago

Thx for reply @nadetastic . I was considering passing on v6 and started doing it in a separate branch, but the project is quite big now, and I have realized it would take a lot. Do you think the SSR limitation for Storage can be fixed in v5 somehow. Maybe we can fix it faster than migrating all our projects to v6.

minatmalek commented 4 months ago

I’m facing the same problem as well. I’m not using Next.js. Same amplify version, apparently it does not refresh the temporary security credentials during upload and then fails after 1hr with an ExpiredToken error. Then I have to refresh the page to get new temporary security credentials.

kolodi commented 4 months ago

I ended up migrating to v6 and not using ssr

minatmalek commented 4 months ago

I ended up migrating to v6 and not using ssr

I upgraded to v6 today, but I'm still getting an "ExpiredToken" error after exactly one hour when uploading files. I'm not using ssr as well.

My investigation suggests that the temporary security credentials used for uploading aren't being refreshed. These credentials expire after one hour. The code seems to check the token's expiration before each upload part, but incorrectly logs that it's still valid even when it's not.

Some Logs:

Then it stops completely.

EmmanuelRTM commented 1 month ago

It's suggested at Expired Token: The Token is Expired - Managing Tokens Properly #12734 that we should migrate from Amplify V5 to V6.