search

aws-amplify / amplify-cli

The AWS Amplify CLI is a toolchain for simplifying serverless web and mobile development.
Apache License 2.0
2.81k stars 821 forks source link

Storage.put / Storage.get attempting to use localhost bucket not deployed AWS S3 bucket #11493

Open orangegrove1955 opened 2 years ago

orangegrove1955 commented 2 years ago

Before opening, please confirm:

JavaScript Framework

React

Amplify APIs

Authentication, GraphQL API, Storage

Amplify Categories

No response

Environment information

``` # Put output below this line System: OS: macOS 11.6 CPU: (4) x64 Intel(R) Core(TM) i7-6567U CPU @ 3.30GHz Memory: 1016.43 MB / 16.00 GB Shell: 3.2.57 - /bin/bash Binaries: Node: 16.10.0 - ~/.nvm/versions/node/v16.10.0/bin/node npm: 7.24.0 - ~/.nvm/versions/node/v16.10.0/bin/npm Browsers: Chrome: 94.0.4606.81 Safari: 15.0 npmPackages: @aws-amplify/ui-react: ^1.2.14 => 1.2.20 @testing-library/jest-dom: ^5.14.1 => 5.14.1 @testing-library/react: ^11.2.7 => 11.2.7 @testing-library/user-event: ^12.8.3 => 12.8.3 aws-amplify: ^4.3.2 => 4.3.2 aws-appsync: ^4.1.1 => 4.1.2 grommet: ^2.17.5 => 2.18.0 grommet-icons: ^4.6.2 => 4.6.2 react: ^17.0.2 => 17.0.2 react-dom: ^17.0.2 => 17.0.2 react-dropzone: ^11.3.4 => 11.4.2 react-router-dom: ^5.3.0 => 5.3.0 react-scripts: 4.0.3 => 4.0.3 styled-components: ^5.3.1 => 5.3.1 styled-components/macro: undefined () styled-components/native: undefined () styled-components/primitives: undefined () uuid: ^8.3.2 => 8.3.2 (3.4.0, 3.3.2) web-vitals: ^1.1.2 => 1.1.2 npmGlobalPackages: @aws-amplify/cli: 6.3.0 corepack: 0.9.0 npm: 7.24.0 ```

Describe the bug

Using Storage.put I am receiving an ERR_CONNECTION_REFUSED response on the Amplify hosted version of my React app. I am also seeing this same issue in my local version of the React app when amplify mock is not running

The link address in the console errors is http://localhost:20005/XXXXXX-dev/public/XXXXXX?x-id=PutObject (XXXXXXX used in place of specific details, let me know if you need them) which seems like the online hosted version is trying to use the locally mocked S3 bucket details instead of the actual bucket online

When using Storage.get, it also appears to use the localhost value for the link address

I have seen issues aws-amplify/amplify-js#5608 aws-amplify/amplify-js#5572 aws-amplify/amplify-js#5621 and tried the suggested method of removing the package-lock.json file and node_modules/ folder, but this has not resolved the issue

Expected behavior

I would expect the Storage.put and Storage.get calls to use the deployed S3 bucket not localhost, and successfully put files into the S3 bucket

Reproduction steps

  1. Add storage to amplify project as per Amplify docs - https://docs.amplify.aws/lib/storage/getting-started/q/platform/js/
  2. Use default values for all CLI requests when asked to add auth
  3. Use Auth and guest users for bucket permissions in CLI
  4. Give all access to auth users
  5. Give read only access to guest users
  6. Deploy using amplify push
  7. Create new user in Cognito
  8. Start amplify mock
  9. Start app with npm run start
    1. Log in with AmplifyAuthenticator
  11. Attempt to use Storage.put locally using amplify mock - should work
  12. Stop using amplify mock
  13. Attempt to use Storage.put on S3 bucket
  14. View console for ERR:CONNECTION_REFUSED with a localhost based link

Code Snippet

// Put your code below this line.
import React from "react";
import { AmplifyAuthenticator } from "@aws-amplify/ui-react";
import { Auth, Storage } from "aws-amplify";

function App() {

  const logOut = async () => {
    console.log(`Logging out`);
    try {
      await Auth.signOut();
      console.log("Logged out");
    } catch (error) {
      console.log(`Error logging out: ${error}`);
    }
  };

  const getStuff = async () => {
    console.log('get')
    try {
      const fileResponse = await Storage.get('test.txt');
      console.log(fileResponse);
    } catch (error) {
      console.error('error happened', error);
    }
  }

  const listStuff = async () => {
    console.log('list')
    try {
      const fileResponse = await Storage.list('/');
      setImages(fileResponse);
    } catch (error) {
      console.error('error happened', error);
    }
  }
  const putStuff = async () => {
    console.log('put')
    try {
      const fileResponse = await Storage.put('test.txt', 'test.txt');
      console.log(fileResponse);
    } catch (error) {
      console.error('error happened', error);
    }
  }

  return (
    <AmplifyAuthenticator hideSignUp={true}>
      {/* <> */}
      <button onClick={putStuff}>Put</button>
      <button onClick={getStuff}>Get</button>
      <button onClick={listStuff}>List</button>

      <button onClick={logOut}>Log Out</button>
    </AmplifyAuthenticator>
  );
}

export default App;

Log output

``` // Put your logs below this line Error: Network Error at createError (createError.js:16) at XMLHttpRequest.handleError (xhr.js:99) uploadImages.js:158 {"message":"Network Error","name":"Error","stack":"Error: Network Error\n at createError (http://localhost:3000/static/js/vendors~main.chunk.js:387044:15)\n at XMLHttpRequest.handleError (http://localhost:3000/static/js/vendors~main.chunk.js:386491:14)","config":{"url":"http://localhost:20005/XXXXXX-dev/public/XXXXXX?x-id=PutObject","method":"put","data":{"path":"IMG_3720.JPG"},"headers":{"Accept":"application/json, text/plain, */*","Content-Type":"binary/octet-stream","x-amz-user-agent":"aws-sdk-js/3.6.1 os/macOS/10.15.7 lang/js md/browser/Chrome_93.0.4577.82 api/s3/3.6.1 aws-amplify/4.3.1_js","amz-sdk-invocation-id":"XXXX","amz-sdk-request":"attempt=1; max=3","x-amz-date":"20211017T043637Z","x-amz-security-token":"XXXX","x-amz-content-sha256":"UNSIGNED-PAYLOAD","authorization":"AWS4-HMAC-SHA256 Credential=XXXX, SignedHeaders=XXXX, Signature=XXXX"},"transformRequest":[null],"transformResponse":[null],"timeout":0,"responseType":"blob","xsrfCookieName":"XSRF-TOKEN","xsrfHeaderName":"X-XSRF-TOKEN","maxContentLength":-1,"maxBodyLength":-1,"cancelToken":{"promise":{}},"transitional":{"silentJSONParsing":true,"forcedJSONParsing":true,"clarifyTimeoutError":false}}} ```

aws-exports.js

const awsmobile = {
    "aws_project_region": "ap-southeast-2",
    "aws_appsync_graphqlEndpoint": "https://XXXX.appsync-api.ap-southeast-2.amazonaws.com/graphql",
    "aws_appsync_region": "ap-southeast-2",
    "aws_appsync_authenticationType": "AMAZON_COGNITO_USER_POOLS",
    "aws_appsync_apiKey": "XXXX",
    "aws_appsync_dangerously_connect_to_http_endpoint_for_testing": true,
    "aws_cognito_identity_pool_id": "ap-southeast-2:XXXX",
    "aws_cognito_region": "ap-southeast-2",
    "aws_user_pools_id": "ap-southeast-2_XXXX",
    "aws_user_pools_web_client_id": "XXXX",
    "oauth": {},
    "aws_cognito_login_mechanisms": [
        "EMAIL"
    ],
    "aws_cognito_signup_attributes": [
        "EMAIL"
    ],
    "aws_cognito_mfa_configuration": "OFF",
    "aws_cognito_mfa_types": [
        "SMS"
    ],
    "aws_cognito_password_protection_settings": {
        "passwordPolicyMinLength": 8,
        "passwordPolicyCharacters": []
    },
    "aws_cognito_verification_mechanisms": [
        "EMAIL"
    ],
    "aws_user_files_s3_bucket": "XXXX-dev",
    "aws_user_files_s3_bucket_region": "ap-southeast-2",
    "aws_user_files_s3_dangerously_connect_to_http_endpoint_for_testing": true,
    "aws_cognito_login_mechanism": [
        "EMAIL"
    ]
};

export default awsmobile;

Manual configuration

No response

Additional configuration

No response

Mobile Device

No response

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

Screen Shot 2021-10-17 at 4 13 31 pm
chrisbonifacio commented 2 years ago

Hi @orangegrove1955 Thanks for raising this issue. I was able to reproduce this and it seems when stopping the mock endpoint, this line that is added by mocking remains in the aws-exports file.

"aws_user_files_s3_dangerously_connect_to_http_endpoint_for_testing": true

A temporary workaround may be to manually remove it in order to go back to using the deployed S3 bucket.

Because the mock functionality belongs to the CLI, I'm going to transfer this issue over to the CLI repo to be looked into further.

josefaidt commented 2 years ago

Hey @orangegrove1955 just to echo what @chrisbonifacio said, thanks for raising 🙂. I've also confirmed I am seeing the aws-exports line get added when we start mock, and when we stop mock the setting remains which can cause confusion. Ideally this setting will be set to false or removed all together when stopping the mock functionality. Marking as a bug 🙂

orangegrove1955 commented 2 years ago

Hey, thanks for the fix! It does seem to have fixed the location the PUT call is being made to, it's aimed at the bucket now. It appears I'm getting a CORS issue here now though, any suggestions on how to resolve them?

Thanks in advance!

Screen Shot 2021-10-19 at 10 29 43 am
orangegrove1955 commented 2 years ago

Hey, sorry, further update after more digging. Turns out the bucket that was being used in aws-exports as well as amplify/backend/storage/myStorage/parameters.json didn't match up with the one in my AWS console

Specifically, the bucket that exists in my AWS console is myStorageBucket203122 but the name in both the files is just myStorageBucket (this part of the name is changed, but not the numbers, let me know if you need it specifically)

I don't recall any point where I renamed the bucket or changed these files, so not sure how this has happened, but wanted to put it down here for your information, and for anyone else who might encounter this issue