[ReferenceError: XMLHttpRequest is not defined] if using validateObjectExistence in the Server version of getUrl function

vmurin commented 2 months ago

Before opening, please confirm:

[X] I have searched for duplicate or closed issues and discussions.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

JavaScript Framework

Next.js

Amplify APIs

Storage

Amplify Version

v6

Amplify Categories

storage

Backend

Amplify Gen 2 (Preview)

Environment information

``` # Put output below this line System: OS: Linux 5.15 Ubuntu 22.04.4 LTS 22.04.4 LTS (Jammy Jellyfish) CPU: (8) x64 Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz Memory: 11.81 GB / 15.50 GB Container: Yes Shell: 5.1.16 - /bin/bash Binaries: Node: 20.10.0 - ~/.nvm/versions/node/v20.10.0/bin/node Yarn: 1.22.22 - ~/.nvm/versions/node/v20.10.0/bin/yarn npm: 10.5.2 - ~/.nvm/versions/node/v20.10.0/bin/npm bun: 1.0.29 - ~/.bun/bin/bun npmPackages: %name%: 0.1.0 @ampproject/toolbox-optimizer: undefined () @aws-amplify/adapter-nextjs: ^1.2.5 => 1.2.5 @aws-amplify/adapter-nextjs/api: undefined () @aws-amplify/adapter-nextjs/data: undefined () @aws-amplify/backend: ^1.0.2 => 1.0.3 @aws-amplify/backend-cli: ^1.0.3 => 1.0.4 @aws-amplify/ui-react: ^6.1.12 => 6.1.12 @aws-amplify/ui-react-internal: undefined () @aws-amplify/ui-react-storage: ^3.1.3 => 3.1.3 @babel/core: undefined () @babel/runtime: 7.22.5 @chakra-ui/icons: ^2.1.1 => 2.1.1 @chakra-ui/next-js: ^2.2.0 => 2.2.0 @chakra-ui/react: ^2.8.2 => 2.8.2 @edge-runtime/cookies: 4.1.1 @edge-runtime/ponyfill: 2.4.2 @edge-runtime/primitives: 4.1.0 @emotion/react: ^11.11.4 => 11.11.4 @emotion/styled: ^11.11.5 => 11.11.5 @hapi/accept: undefined () @mswjs/interceptors: undefined () @napi-rs/triples: undefined () @next/font: undefined () @opentelemetry/api: undefined () @types/node: ^20 => 20.14.8 @types/react: ^18 => 18.3.3 @types/react-dom: ^18 => 18.3.0 @vercel/nft: undefined () @vercel/og: 0.6.2 acorn: undefined () amphtml-validator: undefined () anser: undefined () arg: undefined () assert: undefined () async-retry: undefined () async-sema: undefined () aws-amplify: ^6.3.7 => 6.3.7 aws-amplify/adapter-core: undefined () aws-amplify/analytics: undefined () aws-amplify/analytics/kinesis: undefined () aws-amplify/analytics/kinesis-firehose: undefined () aws-amplify/analytics/personalize: undefined () aws-amplify/analytics/pinpoint: undefined () aws-amplify/api: undefined () aws-amplify/api/server: undefined () aws-amplify/auth: undefined () aws-amplify/auth/cognito: undefined () aws-amplify/auth/cognito/server: undefined () aws-amplify/auth/enable-oauth-listener: undefined () aws-amplify/auth/server: undefined () aws-amplify/data: undefined () aws-amplify/data/server: undefined () aws-amplify/datastore: undefined () aws-amplify/in-app-messaging: undefined () aws-amplify/in-app-messaging/pinpoint: undefined () aws-amplify/push-notifications: undefined () aws-amplify/push-notifications/pinpoint: undefined () aws-amplify/storage: undefined () aws-amplify/storage/s3: undefined () aws-amplify/storage/s3/server: undefined () aws-amplify/storage/server: undefined () aws-amplify/utils: undefined () aws-cdk: ^2.143.0 => 2.147.0 aws-cdk-lib: ^2.143.0 => 2.147.0 babel-packages: undefined () browserify-zlib: undefined () browserslist: undefined () buffer: undefined () bytes: undefined () ci-info: undefined () cli-select: undefined () client-only: 0.0.1 commander: undefined () comment-json: undefined () compression: undefined () conf: undefined () constants-browserify: undefined () constructs: ^10.3.0 => 10.3.0 content-disposition: undefined () content-type: undefined () cookie: undefined () cross-spawn: undefined () crypto-browserify: undefined () css.escape: undefined () data-uri-to-buffer: undefined () debug: undefined () devalue: undefined () domain-browser: undefined () edge-runtime: undefined () esbuild: ^0.21.3 => 0.21.5 eslint: ^8 => 8.57.0 eslint-config-next: 14.2.3 => 14.2.3 events: undefined () find-cache-dir: undefined () find-up: undefined () framer-motion: ^11.2.6 => 11.2.11 fresh: undefined () get-orientation: undefined () glob: undefined () gzip-size: undefined () http-proxy: undefined () http-proxy-agent: undefined () https-browserify: undefined () https-proxy-agent: undefined () icss-utils: undefined () ignore-loader: undefined () image-size: undefined () is-animated: undefined () is-docker: undefined () is-wsl: undefined () jest-worker: undefined () json5: undefined () jsonwebtoken: undefined () loader-runner: undefined () loader-utils: undefined () lodash.curry: undefined () lru-cache: undefined () mini-css-extract-plugin: undefined () nanoid: undefined () native-url: undefined () neo-async: undefined () next: 14.2.3 => 14.2.3 node-fetch: undefined () node-html-parser: undefined () ora: undefined () os-browserify: undefined () p-limit: undefined () path-browserify: undefined () picomatch: undefined () platform: undefined () postcss: ^8 => 8.4.38 (8.4.31) postcss-flexbugs-fixes: undefined () postcss-modules-extract-imports: undefined () postcss-modules-local-by-default: undefined () postcss-modules-scope: undefined () postcss-modules-values: undefined () postcss-preset-env: undefined () postcss-safe-parser: undefined () postcss-scss: undefined () postcss-value-parser: undefined () process: undefined () punycode: undefined () querystring-es3: undefined () raw-body: undefined () react: ^18 => 18.3.1 react-builtin: undefined () react-dom: ^18 => 18.3.1 react-dom-builtin: undefined () react-dom-experimental-builtin: undefined () react-experimental-builtin: undefined () react-is: 18.2.0 react-refresh: 0.12.0 react-server-dom-turbopack-builtin: undefined () react-server-dom-turbopack-experimental-builtin: undefined () react-server-dom-webpack-builtin: undefined () react-server-dom-webpack-experimental-builtin: undefined () regenerator-runtime: 0.13.4 sass-loader: undefined () scheduler-builtin: undefined () scheduler-experimental-builtin: undefined () schema-utils: undefined () semver: undefined () send: undefined () server-only: 0.0.1 setimmediate: undefined () shell-quote: undefined () source-map: undefined () source-map08: undefined () stacktrace-parser: undefined () stream-browserify: undefined () stream-http: undefined () string-hash: undefined () string_decoder: undefined () strip-ansi: undefined () superstruct: undefined () tailwindcss: ^3.4.1 => 3.4.4 tar: undefined () terser: undefined () text-table: undefined () timers-browserify: undefined () tsx: ^4.11.0 => 4.15.7 tty-browserify: undefined () typescript: ^5.4.5 => 5.5.2 (4.4.4, 4.9.5) ua-parser-js: undefined () unistore: undefined () util: undefined () vm-browserify: undefined () watchpack: undefined () web-vitals: undefined () webpack: undefined () webpack-sources: undefined () ws: undefined () xior: ^0.5.0 => 0.5.1 zod: undefined () npmGlobalPackages: @vue/cli: 5.0.8 corepack: 0.22.0 npm: 10.5.2 serverless: 3.38.0 yarn: 1.22.22 ```

Describe the bug

If using the option validateObjectExistence in server function getUrl following error is thrown:

[ReferenceError: XMLHttpRequest is not defined]

Unfortunately error is thrown not on every call See complete code snippet below

Expected behavior

server version of the getUrl do NOT use XMLHttpRequest (see stack trace below) and returns the URL of the S3 object

Reproduction steps

The code snippet is from the following repo: https://github.com/vmurin/crypto-portfolio

File: https://github.com/vmurin/crypto-portfolio/blob/master/app/utils/utils.ts

check out branch amplify-issue
npm install
npm run dev
open localhost:3000
create new account and login
upload example file with name cryptoData.json from README
open home page and check the errors

Code Snippet

// Put your code below this line.
const { runWithAmplifyServerContext } = createServerRunner({
  config,
});

export const getCoinsFromS3 = async () => 
  await runWithAmplifyServerContext({
    nextServerContext: { cookies },
    async operation(contextSpec) {
      const fileUrl = await getUrl(contextSpec, {
        path: ({identityId}) => `coinlists/${identityId}/cryptoData.json`,
        options: {
          validateObjectExistence: true,  // defaults to false
        },
      });
      const result = await fetch(fileUrl.url.toString());
      if (result.ok) {
        return result.json();
      } else {
        throw new Error(`Status: ${result.status} Error: ${result.statusText}`)
      }
    }
  });

Log output

``` // Put your logs below this line getCoinsFromS3 Error: [ReferenceError: XMLHttpRequest is not defined] ⨯ Error [ReferenceError]: XMLHttpRequest is not defined at eval (webpack-internal:///(rsc)/./node_modules/@aws-amplify/storage/dist/esm/providers/s3/utils/client/runtime/xhrTransferHandler.mjs:30:19) at new Promise () at xhrTransferHandler (webpack-internal:///(rsc)/./node_modules/@aws-amplify/storage/dist/esm/providers/s3/utils/client/runtime/xhrTransferHandler.mjs:29:12) at composedHandler (webpack-internal:///(rsc)/./node_modules/@aws-amplify/core/dist/esm/clients/internal/composeTransferHandler.mjs:17:54) at signingMiddleware (webpack-internal:///(rsc)/./node_modules/@aws-amplify/core/dist/esm/clients/middleware/signing/middleware.mjs:34:32) at async retryMiddleware (webpack-internal:///(rsc)/./node_modules/@aws-amplify/core/dist/esm/clients/middleware/retry/middleware.mjs:32:28) at async userAgentMiddleware (webpack-internal:///(rsc)/./node_modules/@aws-amplify/core/dist/esm/clients/middleware/userAgent/middleware.mjs:24:30) at async eval (webpack-internal:///(rsc)/./node_modules/@aws-amplify/core/dist/esm/clients/internal/composeServiceApi.mjs:20:26) at async getProperties (webpack-internal:///(rsc)/./node_modules/@aws-amplify/storage/dist/esm/providers/s3/apis/internal/getProperties.mjs:49:22) at async getUrl (webpack-internal:///(rsc)/./node_modules/@aws-amplify/storage/dist/esm/providers/s3/apis/internal/getUrl.mjs:51:9) ```

aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

No response

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

No response

HuiSF commented 2 months ago

Hi @vmurin Thank you for opening up this issue. I'm unable to reproduce this issue from quick testing, and I was actually seeing it use the fetch API under the hood. The repo you shared seems to be a provided repo, can you grant me access to it?

vmurin commented 2 months ago

Hi @HuiSF, I sent you an invitation to the repo

HuiSF commented 2 months ago

Hey @vmurin Thanks for provide the sample code and detailed instruction. I was able to reproduce the same error testing with the API route /fetch-prices. Noticeably you specified to use EdgeRuntime on this route, that may actually specified the bundle built for this API Route runs as a "browser" bundle. In the storage package, we are rerouting to use XHR instead of fetch on purpose to support better upload progress tracking.

https://github.com/aws-amplify/amplify-js/blob/bca15d7d4750ac786d798b9de88461c187348e34/packages/storage/package.json#L15

Next.js is building bundle for EdgeRuntime with browser, details see: https://github.com/vercel/next.js/discussions/41235 Hence, we have not listed any Storage API as supported server-side API on Next.js EdgeRuntime. See this documentation for details.

vmurin commented 2 months ago

Hey @HuiSF Thank you very much for your analysis. Exactly - that was the case - on one route the code was running as intended and on on the /fetch-price the error was thrown. And I was wondering what makes the difference! No, EdgeRuntime was just an experiment and has no any background. :) You are absolutely right - the storage API ist not supported on the Edge... I was not realizing that I have it activated :)

HuiSF commented 2 months ago

No worries @vmurin glad that this helps. Please let us know if anything else we can assist

aws-amplify / amplify-js