amplify mock storage gives 404 in flutter

[benwimpory]

Description

Flutter s3 storage plugin throws 404 exceptions when uploading or downloading files when using amplify mock storage on ios simulator.

If I close the amplify mock storage process and re-run against a real AWS endpoint, S3 uploads and downloads work perfectly from my flutter app.

In both cases, the app is successfully authenticating with a real cognito backend.

I have done an amplify push, edited the Info.plist file as per the amplify mocking documentation.

The flutter exception I get when either uploading or downloading is the same.

flutter: Error downloading file: StorageException(message: The key 'public/testkey' could not be found with message: Received HTTP Response status code 404 NotFound., recoverySuggestion: Make sure the key exists before trying to download it.
The object for key in the public access level should exist under the public folder as public/<key>.
When looking for the key in protected or private access level, it will be under its respective folder
such as 'protected/<targetIdentityId>/<key>' or 'private/<targetIdentityId>/<key>'., underlyingException: null)

Categories

• [ ] Analytics
• [ ] API (REST)
• [ ] API (GraphQL)
• [ ] Auth
• [ ] Authenticator
• [ ] DataStore
• [X] Storage

Steps to Reproduce

• Create a flutter app that uploads and downloads to S3 as per documentation.
• Run app and watch it successfully upload and download to real AWS S3 endpoint.
• Run amplify mock storage
• Receive a 404 when app tries to upload or download

Screenshots

No response

Platforms

• [X] iOS
• [ ] Android
• [ ] Web
• [ ] macOS
• [ ] Windows
• [ ] Linux

Android Device/Emulator API Level

No response

Environment

Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.3.9, on macOS 12.2.1 21D62 darwin-arm, locale en-GB)
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 13.4.1)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2021.2)
[✓] IntelliJ IDEA Ultimate Edition (version 2022.2.1)
[✓] IntelliJ IDEA Community Edition (version 2022.2.2)
[✓] VS Code (version 1.73.1)
[✓] Connected device (4 available)
[✓] HTTP Host Availability

• No issues found!

Dependencies

Dart SDK 2.18.5
Flutter SDK 3.3.9
aaa 1.0.0+1

dependencies:
- amplify_api 0.6.10 [amplify_api_android amplify_api_ios amplify_core amplify_flutter aws_common collection flutter meta plugin_platform_interface]
- amplify_auth_cognito 0.6.10 [amplify_auth_cognito_android amplify_auth_cognito_ios amplify_core aws_common collection flutter meta plugin_platform_interface]
- amplify_flutter 0.6.10 [amplify_core amplify_datastore_plugin_interface amplify_flutter_android amplify_flutter_ios aws_common collection flutter meta plugin_platform_interface]
- amplify_storage_s3 0.6.10 [amplify_storage_s3_android amplify_storage_s3_ios amplify_core aws_common flutter meta plugin_platform_interface path_provider path]
- beamer 1.5.3 [flutter flutter_web_plugins]
- cupertino_icons 1.0.5
- data_table_2 2.3.8 [flutter async]
- flutter 0.0.0 [characters collection material_color_utilities meta vector_math sky_engine]
- graphql 5.1.1 [meta path gql gql_exec gql_link gql_http_link gql_transform_link gql_error_link gql_dedupe_link hive normalize http collection web_socket_channel stream_channel rxdart uuid]
- intl 0.17.0 [clock path]
- isar 3.0.5 [ffi js meta]
- isar_flutter_libs 3.0.5 [flutter isar]
- path_provider 2.0.11 [flutter path_provider_android path_provider_ios path_provider_linux path_provider_macos path_provider_platform_interface path_provider_windows]
- provider 6.0.3 [collection flutter nested]
- retrieval 1.0.1 [characters]
- rxdart 0.27.5
- safeprint 0.0.2 [flutter intl]

transitive dependencies:
- amplify_api_android 0.6.10 [flutter]
- amplify_api_ios 0.6.10 [amplify_core flutter]
- amplify_auth_cognito_android 0.6.10 [flutter]
- amplify_auth_cognito_ios 0.6.10 [amplify_core flutter]
- amplify_core 0.6.10 [aws_common collection flutter intl json_annotation meta plugin_platform_interface uuid]
- amplify_datastore_plugin_interface 0.6.10 [amplify_core collection flutter meta]
- amplify_flutter_android 0.6.10 [flutter]
- amplify_flutter_ios 0.6.10 [amplify_core flutter]
- amplify_storage_s3_android 0.6.10 [flutter]
- amplify_storage_s3_ios 0.6.10 [flutter]
- async 2.9.0 [collection meta]
- aws_common 0.1.1 [async collection http meta stream_transform uuid]
- characters 1.2.1
- clock 1.1.1
- collection 1.16.0
- crypto 3.0.2 [typed_data]
- ffi 2.0.1
- file 6.1.2 [meta path]
- flutter_web_plugins 0.0.0 [flutter js characters collection material_color_utilities meta vector_math]
- gql 0.13.1 [collection meta source_span]
- gql_dedupe_link 2.0.2 [async gql_exec gql_link meta]
- gql_error_link 0.2.2 [async gql_exec gql_link meta]
- gql_exec 0.4.0 [collection gql meta]
- gql_http_link 0.4.2 [gql gql_exec gql_link http http_parser meta]
- gql_link 0.4.2 [gql gql_exec meta]
- gql_transform_link 0.2.2 [gql_exec gql_link]
- hive 2.2.3 [meta crypto]
- http 0.13.5 [async http_parser meta path]
- http_parser 4.0.1 [collection source_span string_scanner typed_data]
- js 0.6.4
- json_annotation 4.7.0 [meta]
- material_color_utilities 0.1.5
- meta 1.8.0
- nested 1.0.0 [flutter]
- normalize 0.6.0+1 [gql collection]
- path 1.8.2
- path_provider_android 2.0.22 [flutter path_provider_platform_interface]
- path_provider_ios 2.0.11 [flutter path_provider_platform_interface]
- path_provider_linux 2.1.7 [ffi flutter path path_provider_platform_interface xdg_directories]
- path_provider_macos 2.0.6 [flutter path_provider_platform_interface]
- path_provider_platform_interface 2.0.5 [flutter platform plugin_platform_interface]
- path_provider_windows 2.1.3 [ffi flutter path path_provider_platform_interface win32]
- platform 3.1.0
- plugin_platform_interface 2.1.3 [meta]
- process 4.2.4 [file path platform]
- sky_engine 0.0.99
- source_span 1.9.0 [collection path term_glyph]
- stream_channel 2.1.0 [async]
- stream_transform 2.0.0
- string_scanner 1.1.1 [source_span]
- term_glyph 1.2.1
- typed_data 1.3.1 [collection]
- uuid 3.0.6 [crypto]
- vector_math 2.1.2
- web_socket_channel 2.2.0 [async crypto stream_channel]
- win32 3.1.2 [ffi]
- xdg_directories 0.2.0+2 [meta path process]

Device

iphone 13

OS

ios 15.1

Deployment Method

Amplify CLI

CLI Version

10.5.1

Additional Context

No response

Amplify Config

const amplifyconfig = ''' {
    "UserAgent": "aws-amplify-cli/2.0",
    "Version": "1.0",
    "api": {
        "plugins": {
            "awsAPIPlugin": {
                "aaa": {
                    "endpointType": "GraphQL",
                    "endpoint": "https://aaaa.eu-west-1.amazonaws.com/graphql",
                    "region": "eu-west-1",
                    "authorizationType": "API_KEY",
                    "apiKey": "da2-aaa"
                }
            }
        }
    },
    "auth": {
        "plugins": {
            "awsCognitoAuthPlugin": {
                "UserAgent": "aws-amplify-cli/0.1.0",
                "Version": "0.1.0",
                "IdentityManager": {
                    "Default": {}
                },
                "AppSync": {
                    "Default": {
                        "ApiUrl": "https://aaa.appsync-api.eu-west-1.amazonaws.com/graphql",
                        "Region": "eu-west-1",
                        "AuthMode": "API_KEY",
                        "ApiKey": "da2-aaa",
                        "ClientDatabasePrefix": "aaa_API_KEY",
                        "DangerouslyConnectToHTTPEndpointForTesting": true
                    }
                },
                "CredentialsProvider": {
                    "CognitoIdentity": {
                        "Default": {
                            "PoolId": "eu-west-1:39337e51-d361-4295-8906-43ebde0aaa59",
                            "Region": "eu-west-1"
                        }
                    }
                },
                "CognitoUserPool": {
                    "Default": {
                        "PoolId": "eu-west-1_aaa",
                        "AppClientId": "aaa",
                        "Region": "eu-west-1"
                    }
                },
                "Auth": {
                    "Default": {
                        "authenticationFlowType": "USER_SRP_AUTH",
                        "socialProviders": [],
                        "usernameAttributes": [
                            "EMAIL"
                        ],
                        "signupAttributes": [
                            "EMAIL"
                        ],
                        "passwordProtectionSettings": {
                            "passwordPolicyMinLength": 8,
                            "passwordPolicyCharacters": []
                        },
                        "mfaConfiguration": "OFF",
                        "mfaTypes": [
                            "SMS"
                        ],
                        "verificationMechanisms": [
                            "EMAIL"
                        ]
                    }
                },
                "S3TransferUtility": {
                    "Default": {
                        "Bucket": "aaabucket-dev",
                        "Region": "eu-west-1",
                        "DangerouslyConnectToHTTPEndpointForTesting": true
                    }
                }
            }
        }
    },
    "storage": {
        "plugins": {
            "awsS3StoragePlugin": {
                "bucket": "aaa-dev",
                "region": "eu-west-1",
                "defaultAccessLevel": "guest"
            }
        }
    }
}''';

[HuiSF]

Hi @benwimpory thanks for opening this issue. amplify mock storage doesn't work with amplfy-flutter unfortunately.

I did some digging of how amplify-js and iOS, Android work with the mock, it looks like amplify-flutter missing a piece to detect the mock mode and override underlying service API endpoint with the local mock endpoint, and Amplify CLI cannot modify amplifyconfiguration.dart correctly atm.

Would missing local mock capability be a blocker for your work while Amplify offers developer environment support which should be able to cover general development cycle.

[benwimpory]

Thanks for the quick reply on this.

It's good to have had confirmation this is an unsupported feature rather than me doing something wrong! A small note on the documentation may save others hours of time trying to make this feature work.

It's not an outright blocker but defintely an inconvenience as I travel a lot and am often without a stable internet connection so cannot use developer environment support. It's also far quicker to developer using mocks (as I've found with GQL API mocking) due to latency. I can write my own fake provider classes to simulate amplify S3 interactions until this issue is fixed.

Would it be possible to extend the flutter s3 plugin to accept the local http storage endpoint URI so I (and others) could manually maintain a separate amplifyconfiguration.dart for local development ? Having the CLI upgraded to automatically switch the configuration would then be a lower priority.

[HuiSF]

Thanks for the follow @benwimpory .

Adding implementation to support local mock in the stable version of the s3 plugin would be cumbersome as the functionality depends on amplify-ios and amplify-android, it should be easier to implement in the next releases which the packages are written in Dart. I'll see how we can prioritize this, and update here if any progress. Thanks for your patience.

[HuiSF]

Summary of items that may be needed to support amplify mock storage:

1. Amplify CLI needs to insert correct local mock override key into amplifyconfiguration.dart on amplify mock storage
2. The Storage S3 plugin needs to be capable to read the override key inserted by Amplify CLI, and to use the local mock endpoint to perform storage operations
3. Allow http with the local mock mode (if this would be a problem blocking testing)

[cwomack]

Related to https://github.com/aws-amplify/amplify-cli/issues/11493 in CLI repo

[josefaidt]

After further review the configurations provided by the CLI are sufficient for the Flutter library to consume and connect to the localhost address. Transferring to Flutter

[dnys1]

We've marked this as a feature request for the Storage library. We will update this ticket when we have more information.