search

aws-amplify / aws-sdk-android

AWS SDK for Android. For more information, see our web site:
https://docs.amplify.aws
Other
1.02k stars 548 forks source link

[Question&Help] Why receive DefaultDataset: Synchronize failed because it exceeded the maximum retries #3515

Closed pengfeizhong closed 4 months ago

pengfeizhong commented 5 months ago

State your question Some times after call dataSet?.synchronize(callback) will print in log DefaultDataset: Synchronize failed because it exceeded the maximum retries and receive Dataset.SyncCallback.onFailure(dse: DataStorageException?) com.amazonaws.mobileconnectors.cognito.exceptions.DataStorageException: Synchronize failed because it exceeded the maximum retries

Not receive any other callback such as "onConflict,onDatasetDeleted,onDatasetsMerged,onSuccess"

Which AWS Services are you utilizing? implementation 'com.amazonaws:aws-android-sdk-cognito:2.16.3' implementation 'com.amazonaws:aws-android-sdk-kinesis:2.16.3' // keep this version will keep core version implementation 'com.amazonaws:aws-android-sdk-s3:2.16.3' implementation 'com.amazonaws:aws-android-sdk-core:2.16.3' Provide code snippets (if applicable) lastSyncDataTime = System.currentTimeMillis() var cost = System.currentTimeMillis() GlobalScope.launch(Dispatchers.IO ) { val callback = object : Dataset.SyncCallback { val datasetDebug: Any? = "Sync" override fun onSuccess( dataset: Dataset?, updatedRecords: MutableList?, ): Unit { datasetDebug?.let { Timber.tag("AWS").e("syncDataSet.onSuccess, update.size=${updatedRecords?.size}") Timber.tag("synchronize-cost").d("cost.callback :${System.currentTimeMillis() - cost}") }

                    if(outerCallback != null) {
                        GlobalScope.launch(Dispatchers.Main) {
                            datasetDebug?.let {
                                Timber.tag("PIN")
                                    .e("outerCallback == null? ${outerCallback == null}")
                            }
                            outerCallback?.onSuccess()
                        }
                    }
                }

                override fun onFailure(dse: DataStorageException?): Unit {
                    datasetDebug?.let {
                        Timber.tag("AWS").e("syncDataSet.onFailure")
                    }
                    if(dse is DataStorageException){
                        Timber.tag("AWS").e("sync count: ${dataSet?.lastSyncCount}")
                    }
                    lastSyncDataTime = 0

                    if(outerCallback != null) {
                        GlobalScope.launch(Dispatchers.Main) {
                            outerCallback?.onFail(dse)
                        }
                    }
                }

                override fun onDatasetDeleted(
                    dataset: Dataset?,
                    datasetName: String?,
                ): Boolean {
                    datasetDebug?.let {
                        Timber.tag("AWS").e("syncDataSet.onDatasetDeleted  ")
                    }
                    return true
                }

                override fun onConflict(
                    dataset: Dataset,
                    conflicts: MutableList<SyncConflict>,
                ): Boolean {
                    val resolved = mutableListOf<Record>()
                    conflicts.forEach {
                        resolved.add(it.resolveWithLastWriterWins())
                        Timber.tag("AWS").e("syncDataSet.onConflict  for key:${it.key} ")
                    }
                    dataset.resolve(resolved)
                    datasetDebug?.let {
                        Timber.tag("AWS").e("syncDataSet.onConflict  ")
                    }
                    /**
                     * return false 不会继续处理冲突，true:会处理完成最终 也会 call onSuccess
                     * if (!callback.onConflict(DefaultDataset.this, conflicts)) {
                    // if they didn't want to continue on resolving conflicts
                    // return
                            return false;
                            }
                     */
                    return true
                }

                override fun onDatasetsMerged(
                    dataset: Dataset?,
                    datasetNames: MutableList<String>?,
                ): Boolean {
                    datasetDebug?.let {
                        Timber.tag("AWS").e("syncDataSet.onDatasetsMerged  ")
                    }
                    return true
                }
            }
            withContext(Dispatchers.IO){
                try {
                    dataSet?.synchronize(callback)
                }catch (ex:Exception){
                    Timber.e("ex : synchronize fail ${ex.localizedMessage}  ${ex.printStackTrace()}")

                    if(outerCallback != null) {
                        lastSyncDataTime =0
                        GlobalScope.launch(Dispatchers.Main) {
                            outerCallback?.onFail(ex)
                        }
                    }
                }
                if(DBG_SYNC_COST){
                    Timber.tag("AWS").d("cost:${System.currentTimeMillis() - cost}")
                }

            }
        }

Environment(please complete the following information):

Device Information (please complete the following information):

If you need help with understanding how to implement something in particular then we suggest that you first look into our developer guide. You can also simplify your process of creating an application, as well as the associated backend setup by using the Amplify CLI.

tylerjroach commented 5 months ago

Can you let me know if you still have the same issue on the latest version of the AWS Android SDK. The version you are using is over 4 years old (Released Nov 2019).

tylerjroach commented 5 months ago

After further research, these classes aren't in the latest version as it was deprecated in favor of AppSync SDK.

https://github.com/aws-amplify/aws-sdk-android/blob/318f759b54b916e9f78a86c7ae3375482066ecc2/aws-android-sdk-cognito/src/main/java/com/amazonaws/mobileconnectors/cognito/DefaultDataset.java#L376

The error comes from here. Synchronize will internally attempt to do so 3 times before giving up. If you turn on the logger for LogFactory.getLog(DefaultDataset.class); for debug, you should be presented with more information why a failure occurred.

pengfeizhong commented 5 months ago

hi, thanks your answer, we will try use newer sdk in future Now logs is very poor, base on the logs, it not trigger onDatasetsMerged, onConflict , so can we think that cause by network connection ? cognito try 3 times and failure trigger this exception?

tylerjroach commented 5 months ago

I would assume it is network related, or possibly auth. Feel free to post additional logs once DefaultDataset logging is set to debug or verbose logging.

pengfeizhong commented 3 months ago

@tylerjroach
Here catch more logs, it looks like Sync Conflict , but Can you give me some help how to let it back to normal work , thanks? " Received error response: com.amazonaws.services.cognitosync.model.ResourceConflictException: Current SyncCount for: history2 is: 5285 not: 5282 (Service: null; Status Code: 409; Error Code: ResourceConflictException; Request ID: 4da19b3c-2e9c-4177-828c-48694e1de7b7) " Full log as below:

03-11 14:54:02.311 3295 3956 D AWS4Signer: AWS4 Canonical Request: '"POST 03-11 14:54:02.311 3295 3956 D AWS4Signer: /identitypools/us-east-1%253A258553b4-b902-43b3-8103-3e6bfd78076e/identities/us-east-1%253A9643cc0e-90bd-4e20-b701-00ba6ef396a6/datasets/com.xxx 03-11 14:54:02.311 3295 3956 D AWS4Signer: 03-11 14:54:02.311 3295 3956 D AWS4Signer: host:cognito-sync.us-east-1.amazonaws.com 03-11 14:54:02.311 3295 3956 D AWS4Signer: x-amz-date:20240311T065402Z 03-11 14:54:02.311 3295 3956 D AWS4Signer: x-amz-security-token:xxx token 03-11 14:54:02.311 3295 3956 D AWS4Signer: 03-11 14:54:02.311 3295 3956 D AWS4Signer: host;x-amz-date;x-amz-security-token 03-11 14:54:02.311 3295 3956 D AWS4Signer: 4cef6484b9d2e6857699df1495b892b70040fb383b80d6c990ae116be0933bc8" 03-11 14:54:02.312 3295 3956 D AWS4Signer: AWS4 String to Sign: '"AWS4-HMAC-SHA256 03-11 14:54:02.312 3295 3956 D AWS4Signer: 20240311T065402Z 03-11 14:54:02.312 3295 3956 D AWS4Signer: 20240311/us-east-1/cognito-sync/aws4_request 03-11 14:54:02.312 3295 3956 D AWS4Signer: 3d501114afa60bbe8f6eac95e086ef00114bba347978d5fa8b086fdf25aea055" 03-11 14:54:02.819 3295 3956 D com.amazonaws.request: Received error response: com.amazonaws.services.cognitosync.model.ResourceConflictException: Current SyncCount for: history2 is: 5285 not: 5282 (Service: null; Status Code: 409; Error Code: ResourceConflictException; Request ID: 4da19b3c-2e9c-4177-828c-48694e1de7b7) 03-11 14:54:02.820 3295 3956 I DefaultDataset: conflicts detected when pushing changes to remote. 03-11 14:54:02.820 3295 3956 E DefaultDataset: Synchronize failed because it exceeded the maximum retries 03-11 14:54:02.820 3295 3956 E APP:AWS: syncDataSet.onFailure 03-11 14:54:02.822 3295 3956 D KeyProvider23: AndroidKeyStore contains keyAlias com.amazonaws.android.auth.aesKeyStoreAlias 03-11 14:54:02.822 3295 3956 D KeyProvider23: Loading the encryption key from Android KeyStore. 03-11 14:54:02.836 3295 3956 D KeyProvider23: AndroidKeyStore contains keyAlias com.amazonaws.android.auth.aesKeyStoreAlias 03-11 14:54:02.836 3295 3956 D KeyProvider23: Loading the encryption key from Android KeyStore. 03-11 14:54:02.852 3295 3956 E APP:AWS: sync count: 5285 03-11 14:54:02.853 3295 3956 D DefaultDataset: failed to synchronize com.xxx 03-11 14:54:02.854 3295 3295 E APP:AWS: syncDataSet resume fail ex:com.amazonaws.mobileconnectors.cognito.exceptions.DataStorageException: Synchronize failed because it exceeded the maximum retries, 1999! 03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: init data set fail:com.amazonaws.mobileconnectors.cognito.exceptions.DataStorageException: Synchronize failed because it exceeded the maximum retries 03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.synchronizeInternal(DefaultDataset.java:377) 03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.handleLocalModifications(DefaultDataset.java:336) 03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.synchronizeInternal(DefaultDataset.java:413) 03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.handleLocalModifications(DefaultDataset.java:336) 03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.synchronizeInternal(DefaultDataset.java:413) 03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.handleLocalModifications(DefaultDataset.java:336) 03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.synchronizeInternal(DefaultDataset.java:413) 03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.handleLocalModifications(DefaultDataset.java:336) 03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.synchronizeInternal(DefaultDataset.java:413) 03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset$1.run(DefaultDataset.java:153) 03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at java.lang.Thread.run(Thread.java:923) 03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: init data set fail:Synchronize failed because it exceeded the maximum retries 03-11 14:54:02.860 3295 3448 E APP:UtilApp:$Companion: step4: get cognito end

this line in log, is in onFailure() callback and exception is DataStorageException

03-11 14:54:02.852 3295 3956 E APP:AWS: sync count: 5285

full callback code as below:

             override fun onFailure(dse: DataStorageException?): Unit {
                    datasetDebug?.let {
                        Timber.tag("AWS").e("syncDataSet.onFailure")
                    }
                    if(dse is DataStorageException){
                        Timber.tag("AWS").e("sync count: ${dataSet?.lastSyncCount}")
                        if(DebugSwitchs.debugPin() && DebugSwitchs.debugFlag()){
                            Timber.tag("PIN")
                                .e("after -------------------  ${dataSet?.lastSyncCount}-")
                            dataSet?.allRecords?.forEach{
                                Timber.tag("PIN")
                                    .e("${it.toString()}")
                            }
                            Timber.tag("PIN")
                                .e("after --------------------")
                        }
                    }

sdk version:

//AWS SDK
implementation 'com.amazonaws:aws-android-sdk-cognito:2.16.3'
implementation 'com.amazonaws:aws-android-sdk-kinesis:2.16.3'  // keep this version  will keep core version
implementation 'com.amazonaws:aws-android-sdk-s3:2.16.3'
implementation 'com.amazonaws:aws-android-sdk-core:2.16.3'