DataStore on iOS with Cognito user pools, websocket protocol error but works with an API key

[Tom-RulesCube]

Which Category is your question related to?

DataStore on iOS

Amplify CLI Version

4.18.0

You can use amplify -v to check the amplify cli version on your system

What AWS Services are you utilizing?

Cognito User Pools API (GraphQL) Storage DataStore various triggers and custom functions

Provide additional details e.g. code snippets

We started on iOS using AppSync API and now want to move onto using DataStore.

Our authorization is based on user pools and groups, and we don't really need/want API Key

When I add DataStore in this configuration, the WebSocket is unable to connect to the GraphQL API end point and I see the following in the iOS log:

2020-04-21 11:30:39.156357-0400 Appname[3544:99327] Connecting to url ...
USER IS SIGNED OUT <-- this is form AWSMobileClient
2020-04-21 11:30:39.262675-0400 Appname[3544:99035] [Snapshotting] Snapshotting a view (0x7ff350c0eb30, UIView) that has not been rendered at least once requires afterScreenUpdates:YES.
2020-04-21 11:30:39.452011-0400 Appname[3544:99324] WebsocketDidConnect
2020-04-21 11:30:39.452147-0400 Appname[3544:99324] WebsocketDidConnect, sending init message...
2020-04-21 11:30:39.452422-0400 Appname[3544:99324] Validating connection
2020-04-21 11:30:39.452626-0400 Appname[3544:99324] WebsocketDidReceiveMessage - {"payload":{"errors":[{"message":"Both, the \"header\", and the \"payload\" query string parameters are missing","errorCode":400}]},"type":"connection_error"}

followed by various messages like:

2020-04-21 11:30:39.466218-0400 Appname[3544:99325] [AWSModelReconciliationQueue] receiveCompletion: error: DataStoreError: subscription item event failed with error
Caused by:
APIError: subscription item event failed with error
Caused by:
jsonParse(nil, Optional(Swift.DecodingError.dataCorrupted(Swift.DecodingError.Context(codingPath: [CodingKeys(stringValue: "type", intValue: nil)], debugDescription: "Cannot initialize RealtimeConnectionProviderResponseType from invalid String value connection_error", underlyingError: nil))))
2020-04-21 11:30:39.466220-0400 Appname[3544:99346] [AWSModelReconciliationQueue] receiveCompletion: error: DataStoreError: subscription item event failed with error

It looks like it's connecting but the payload sent to the endpoint appears to be invalid?

In awsconfiguration.json I see the config for AppSync as follows:

    "AppSync": {
        "Default": {
            "ApiUrl": "https://blahblahblahblahblahblah.appsync-api.us-east-1.amazonaws.com/graphql",
            "Region": "us-east-1",
            "AuthMode": "AMAZON_COGNITO_USER_POOLS",
            "ClientDatabasePrefix": "Appnamecapi_AMAZON_COGNITO_USER_POOLS"
        },
        "Appnamecapi_API_KEY": {
            "ApiUrl": "https://blahblahblahblahblahblahblah.appsync-api.us-east-1.amazonaws.com/graphql",
            "Region": "us-east-1",
            "AuthMode": "API_KEY",
            "ApiKey": "blahblahblahblahblahblah",
            "ClientDatabasePrefix": "Appnamecapi_API_KEY"
        }
    }

So I change this to an API key (but I then have to remove all the @auth directives on my schema--which is not gonna work for the app)

% amplify api update
? Please select from one of the below mentioned services: GraphQL
? Select from the options below Walkthrough all configurations
? Choose the default authorization type for the API API key
? Enter a description for the API key: Appnamecapikey
? After how many days from now the API key should expire (1-365): 365
? Do you want to configure advanced settings for the GraphQL API Yes, I want to make some additional changes.
? Configure additional auth types? No
? Configure conflict detection? Yes
? Select the default resolution strategy Auto Merge
? Do you want to override default per model settings? No

Now awsconfiguration.json has:

   "AppSync": {
        "Default": {
            "ApiUrl": "https://blahblahblahblahblahblah.appsync-api.us-east-1.amazonaws.com/graphql",
            "Region": "us-east-1",
            "AuthMode": "API_KEY",
            "ApiKey": "blahblahblahblahblahblah",
            "ClientDatabasePrefix": "blahblahblahblah_API_KEY"
        }

And now, happiness, and my DataStore operations work (albeit without ownership and groups):

2020-04-21 11:57:35.878735-0400 Appname[4052:122475] Header - Optional("{\"x-amz-date\":\"20200421T155735Z\",\"x-api-key\":\"blahblahblahblahblahblah\",\"host\":\"blahblahblahblahblah.appsync-api.us-east-1.amazonaws.com\"}")
2020-04-21 11:57:35.879285-0400 Appname[4052:122479] Connecting to url ...
2020-04-21 11:57:36.341543-0400 Appname[4052:122475] WebsocketDidConnect
2020-04-21 11:57:36.341634-0400 Appname[4052:122475] WebsocketDidConnect, sending init message...
2020-04-21 11:57:36.341772-0400 Appname[4052:122475] Validating connection
2020-04-21 11:57:36.342174-0400 Appname[4052:122484] Message type does not need signing - connectionInit("connection_init")
2020-04-21 11:57:36.342443-0400 Appname[4052:122484] Websocket write - {"type":"connection_init"}
2020-04-21 11:57:36.374642-0400 Appname[4052:122476] WebsocketDidReceiveMessage - {"type":"connection_ack","payload":{"connectionTimeoutMs":300000}}
2020-04-21 11:57:36.375220-0400 Appname[4052:122476] WebsocketDidReceiveMessage - {"type":"ka"}
2020-04-21 11:57:36.375249-0400 Appname[4052:122475] Start subscription

So, my question: how do I get DataStore to work with Cognito user pools? Is this supported? Config or code?

[Tom-RulesCube]

In addition the init code is:

        try Amplify.add(plugin: AWSAPIPlugin())
        try Amplify.add(plugin: AWSDataStorePlugin(modelRegistration: AmplifyModels()))
        let storagePlugin = AWSS3StoragePlugin()
        try Amplify.add(plugin: storagePlugin)
        try Amplify.configure()

and an AWSMobileCLient extension ...

extension AWSMobileClient: AWSCognitoUserPoolsAuthProviderAsync {
    public func getLatestAuthToken(_ callback: @escaping (String?, Error?) -> Void) {
        getTokens { (tokens, error) in
            if error != nil {
                callback(nil, error)
            } else {
                callback(tokens?.idToken?.tokenString, nil)
            }
        }
    }
}

[Tom-RulesCube]

Ok we can send this question to the Wall of Shame ... of course, you have to have actually logged in with your Cognito pool user before the connection can work. In fairness, I have built websocket APIs before and generally we let you connect before validating what you were trying to do. AppSync won't even talk to you unless you are authorized. Unfortunately the error looks very low level and given that it worked well with the API Key, it did not trigger until I was doing a sequencing step through of the app startup and went aaaaaaahaaaaaaa.

I hooked up the API and DataStore initialization to trigger on the successful login of the AWSMobileClient signup/login and the DataStore websocket connection is suddenly very happy and chatty.

[Tom-RulesCube]

If its helps you ... this is my current initialization code...

    public func initMobile(_ completionHandler: @escaping (Error?) -> Void) {
        _mobile = AWSMobileClient.default()
        _mobile!.addUserStateListener(self) { (userState, info) in
            do {
                switch (userState) {
                case .guest:
                    print("USER IS IN GUEST MODE, INITIALIZE AMPLIFY")
                    try self.initAmplify()
                case .signedOut:
                    print("USER IS SIGNED OUT")
                    try self.doneAmplify()
                case .signedIn:
                    print("USER IS SIGNED IN, INITIALIZE AMPLIFY")
                    try self.initAmplify()
                case .signedOutUserPoolsTokenInvalid:
                    print("USER NEEDS TO LOGIN AGAIN")
                    try self.doneAmplify()
                case .signedOutFederatedTokensInvalid:
                    print("USER LOGGED IN USING FEDERATION BUT NEEDS NEW TOKENS")
                    try self.doneAmplify()
                default:
                    print("UNSUPPORTED")
                    try self.doneAmplify()
                }
            } catch { /* AND IGNORE */ }
        }
        _mobile!.initialize { (userState, error) in
            if error != nil {
                print("ERROR INITIALIZING MOBILE CLIENT")
            }
            completionHandler(error)
        }
    }

[undefobj]

Reopening so that we can track this as a DX improvement for GA. Transferring to iOS team.

[lawmicha]

Hi @Tom-RulesCube thanks for opening this issue. We have added support for using DataStore with Cognito user pools enabled API with some limitations. If you would like to try this feature out while it is still in development, feel free to follow the steps:

1. Please upgrade Amplify CLI to the latest (4.21.0)

   npm install -g @aws-amplify/cli@latest

2. Podfile will need to depend on DataStore, API, and Auth

   pod 'Amplify'
   pod 'AmplifyPlugins/AWSAPIPlugin'
   pod 'AmplifyPlugins/AWSDataStorePlugin'
   pod 'AmplifyPlugins/AWSCognitoAuthPlugin'

   Make sure to run pod install --repo-update too ensure you are picking up the 1.0.0 version

3. the API's auth mode will need to be set to Cognito User Pool. Either amplify add api or amplify update api to set the default auth mode to cognito user pool

   ? Please select from one of the below mentioned services: 
   `GraphQL`
   ? Select from the options below 
   `Update auth settings`
   ? Choose the default authorization type for the API 
   `Amazon Cognito User Pool`

4. What schema are you using? Here's an example that you can use for restricting updates and deletes on your model data by other users. Other users can read the data. all operations are restricted to authenticated cognito user pool users.

type SocialNote
    @model
    @auth(rules: [
        { allow: owner, ownerField: "owner", operations: [create, update, delete] },
    ]) {
    id: ID!
    content: String!
    owner: String
}

5. Provisioning the backend. If using the CLI, run amplify push . if using AmplifyTools, update amplifytools.xcconfig's push=true

6. The configuration file has been consolidated to amplifyconfiguration.json. and there is no need for awsconfiguration.json or a dependency on AWSMobileClient

You will see something like this generated as:

{
    "api": {
        "plugins": {
            "awsAPIPlugin": {
                ....
    "auth": {
        "plugins": {
            "awsCognitoAuthPlugin": {
                ....

7. Configure Auth, DataStore, and API all through Amplify

   do {
   try Amplify.add(plugin: AWSCognitoAuthPlugin())
   try Amplify.add(plugin: AWSAPIPlugin())
   try Amplify.add(plugin: AWSDataStorePlugin(modelRegistration: AmplifyModels()))
   try Amplify.configure()
   print("Initialized Amplify");
   } catch {
   print("Could not initialize Amplify: \(error)")
   }

8. When running the app, DataStore will try to start the sychronization with API and fail since the user is not authenticated. This will be attempted a few times and eventually fail. So there is no need to add a listener for auth events. DataStore.save() will continue to be persist your data to the local store.

9. Sign the user in with Amplify.Auth

   _ = Amplify.Auth.signIn(username: username, password: password) { event in
           switch event {
           case .success:
               print("Sign in successfully")
           case .failure(let error):
               print("Failed to sign in")
           }
       }

10. DataStore will automatically start again when any of the DataStore operations are triggered, such as DataStore.save() or DataStore.query().

[Nicolaidam]

Hi @lawmicha

I also have problems with @auth in my api when using DataStore. I dont really understand your solution. I use amplify 4.21.3 but im still not able to save anything in the cloud and my terminal gets spammed with subscription errors.

Do you simply run DataStore.save() after loggin in successfully? Can you please elaborate your point with 8) ?

[lawmicha]

Hi @Nicolaidam, there are a few authentication requirements to consider when using auth directive enabled API. Sorry, the documentation is still pending release. Once the user is signed in, the data should be successfully synced to and from the API.

When running the app, DataStore will try to start the sychronization with API and fail since the user is not authenticated. This will be attempted a few times and eventually fail. So there is no need to add a listener for auth events. DataStore.save() will continue to be persist your data to the local store.

I forgot to mention that saving data to the cloud will also fail if the user has not signed in, the sync to cloud portion will work only after user has signed in.

Using auth directive does have its limitations as the support for it through DataStore is still in development. What does your schema look like? Feel free to open a new issue with the sample schema that you are trying to use, and any additional logs that you saw while running DataStore, after user has signed in, and still see failures. If you have any questions, we also have a discord channel

[Nicolaidam]

Hi @lawmicha Thanks i will check it out. As a start i need to get DataStore sync to cloud working with my API, but somehow it doesnt work right now. I've created a new issue: https://github.com/aws-amplify/amplify-ios/issues/561

[lawmicha]

closing for now as the feature is in Amplify 1.0.1. feel free to open a new issue if you have any questions

[Tom-RulesCube]

DataStore is working as expected now, and we can see the startup sequence re: Cognito login. In addition, the @auth directive is working fully as expected.

We do see some timeouts with the web socket at various intervals that can make our app appear to go braindead. Will be doing some more testing and if we see problems in Amplify we'll open a new issue.