search

aws-amplify / amplify-android

The fastest and easiest way to use AWS from your Android app.
https://docs.amplify.aws/lib/q/platform/android/
Apache License 2.0
240 stars 112 forks source link

PutObjectOperationDeserializerKt.throwPutObjectError caused by S3Exception - The provided token has expired #2651

Closed yaroslav-v closed 7 months ago

yaroslav-v commented 7 months ago

Before opening, please confirm:

Language and Async Model

Java

Amplify Categories

Authentication, Storage

Gradle script dependencies

```groovy // Put output below this line implementation "com.amplifyframework:core:2.14.5" implementation "com.amplifyframework:aws-storage-s3:2.14.5" implementation "com.amplifyframework:aws-auth-cognito:2.14.5" ```

Environment information

``` # Put output below this line ------------------------------------------------------------ Gradle 8.1.1 ------------------------------------------------------------ Build time: 2023-04-21 12:31:26 UTC Revision: 1cf537a851c635c364a4214885f8b9798051175b Kotlin: 1.8.10 Groovy: 3.0.15 Ant: Apache Ant(TM) version 1.10.11 compiled on July 10 2021 JVM: 1.8.0_201 (Oracle Corporation 25.201-b09) OS: Mac OS X 10.16 x86_64 ```

Please include any relevant guides or documentation you're referencing

https://docs.amplify.aws/android/sdk/auth/

Describe the bug

Hi there!

A part of our users experience issues with uploading images to AWS. The problem has appeared after updating from v1.38.8 to v2.14.5. The issue appears on all Android version from 9 to 14, affected devices include Samsung, Motorola, Oneplus, Google etc.

As far as I can see, this issue appears randomly on part of the devices only. I've tried to reproduce the issue locally but couldn't find exact steps.

You can find a crashlog from Firebase in the attachment. I've checked Firebase reports for the previous versions as well (before the update) and there were no such cases.

The crash itself is caused either by Caused by aws.sdk.kotlin.services.s3.model.S3Exception The provided token has expired.

or by Caused by aws.sdk.kotlin.services.s3.model.S3Exception The provided token is malformed or otherwise invalid.

As far as I understand from the docs and from my previous experience, we don't need to refresh tokens anyhow in this case - "For Guest scenarios they will be automatically refreshed."

Reproduction steps (if applicable)

-

Code Snippet

// Initialization in Application class

Amplify.Logging.disable();

Amplify.addPlugin(new AWSCognitoAuthPlugin());
Amplify.addPlugin(new AWSS3StoragePlugin());

AmplifyConfiguration amplifyConfig = AmplifyConfiguration.builder(context)
        .devMenuEnabled(false)
        .build();
Amplify.configure(amplifyConfig, context);

// Uploading, somewhere in the app

DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ssZ", Locale.US);
dateFormat.setTimeZone(TimeZone.getTimeZone("UTC"));

Map<String, String> metadata = new ArrayMap<>(4);
metadata.put("date", dateFormat.format(date));
metadata.put("session_uuid", uid);
metadata.put("token", authToken);
metadata.put("service", serviceName);

StorageUploadFileOptions options =
        StorageUploadFileOptions.builder()
                .accessLevel(StorageAccessLevel.PUBLIC)
                .contentType(mimeType)
                .metadata(metadata)
                .build();

final File file = new File(path);
if (file.canRead()) {
    Amplify.Storage.uploadFile(
            file.getName(),
            file,
            options,
            result -> {
                // write success logs
            },
            failure -> {
                // write failure logs
            }
    );
}

Log output

``` // Put your logs below this line Non-fatal Exception: com.amplifyframework.storage.StorageException Something went wrong with your AWS S3 Storage upload file operation com.amplifyframework.storage.s3.operation.AWSS3StorageUploadFileOperation$UploadTransferListener.onError (AWSS3StorageUploadFileOperation.kt:228) com.amplifyframework.storage.s3.transfer.TransferStatusUpdater.updateOnError$lambda$11$lambda$10 (TransferStatusUpdater.kt:160) android.os.Handler.handleCallback (Handler.java:978) android.os.Handler.dispatchMessage (Handler.java:104) android.os.Looper.loopOnce (Looper.java:238) android.os.Looper.loop (Looper.java:357) android.app.ActivityThread.main (ActivityThread.java:8098) java.lang.reflect.Method.invoke (Method.java) com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:548) com.android.internal.os.ZygoteInit.main (ZygoteInit.java:1026) Caused by java.lang.Exception aws.sdk.kotlin.services.s3.model.S3Exception: The provided token has expired. com.amplifyframework.storage.s3.transfer.worker.BaseTransferWorker.doWork$suspendImpl (BaseTransferWorker.kt:121) com.amplifyframework.storage.s3.transfer.worker.BaseTransferWorker$doWork$1.invokeSuspend (BaseTransferWorker.kt:11) kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith (ContinuationImpl.kt:33) kotlinx.coroutines.UndispatchedCoroutine.afterResume (CoroutineContext.kt:270) kotlinx.coroutines.AbstractCoroutine.resumeWith (AbstractCoroutine.kt:102) kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith (ContinuationImpl.kt:46) kotlinx.coroutines.UndispatchedCoroutine.afterResume (CoroutineContext.kt:270) kotlinx.coroutines.AbstractCoroutine.resumeWith (AbstractCoroutine.kt:102) kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith (ContinuationImpl.kt:46) kotlinx.coroutines.DispatchedTask.run (DispatchedTask.kt:108) kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely (CoroutineScheduler.kt:584) kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask (CoroutineScheduler.kt:793) kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker (CoroutineScheduler.kt:697) kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run (CoroutineScheduler.kt:684) Caused by aws.sdk.kotlin.services.s3.model.S3Exception The provided token has expired. aws.sdk.kotlin.services.s3.serde.PutObjectOperationDeserializerKt.throwPutObjectError (PutObjectOperationDeserializer.kt:69) aws.sdk.kotlin.services.s3.serde.PutObjectOperationDeserializerKt.access$throwPutObjectError (PutObjectOperationDeserializer.kt:1) aws.sdk.kotlin.services.s3.serde.PutObjectOperationDeserializerKt$throwPutObjectError$1.invokeSuspend (PutObjectOperationDeserializer.kt:10) kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith (ContinuationImpl.kt:33) kotlinx.coroutines.DispatchedTask.run (DispatchedTask.kt:108) kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely (CoroutineScheduler.kt:584) kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask (CoroutineScheduler.kt:793) kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker (CoroutineScheduler.kt:697) kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run (CoroutineScheduler.kt:684) ```

amplifyconfiguration.json

No response

GraphQL Schema

```graphql // Put your schema below this line ```

Additional information and screenshots

No response

gpanshu commented 7 months ago

Can you paste your gradle dependencies here please? My suspicion is that you are trying to use both AWS SDK for Android and Amplify V2. While V1 was dependent on AWS SDK for Android there is no such dependency for Amplify V2.

To add to that you definitely do need to refresh tokens even in guest scenarios, if you do a fetchAuthSession it should automatically do that for you.

yaroslav-v commented 7 months ago

Hi! The dependencies are available in the original post. There are only:

implementation "com.amplifyframework:core:2.14.5"
implementation "com.amplifyframework:aws-storage-s3:2.14.5"
implementation "com.amplifyframework:aws-auth-cognito:2.14.5"

The same dependencies were used with the previous v1.38.8. There were no code changes, no dependencies changes - only the version was updated.

Apart from that, we have some lines in the project build.gradle, but I don't think they make a difference in this case:

buildscript {
    ...
    dependencies {
            classpath 'com.amplifyframework:amplify-tools-gradle-plugin:1.0.1'
            ...
    }
}
...
apply plugin: 'com.amplifyframework.amplifytools'

Btw I did some more tests and currently I can say for sure that the problem appears only if you update the app from the version with AWS Amplify v1.38.8 to the new build with AWS Amplify v2.14.5. If I try to install a fresh new copy with AWS Amplify v2.14.5 all works as expected.

As far as I can see, the problem doesn't appear in a tick of time after updating the app, but after some period. I'm not sure about the exact length, but looks like it's something about 1h.

Anyway, a fresh new installation doesn't have the described issue and works well without any changes for longer periods of time. I've tried it for a couple of days and there were no such problems.

gpanshu commented 7 months ago

Yes that makes sense as the refresh token is expired in that case. Can you try one more thing and advise what happens in the following scenario:

  1. You update your app from 1.38.8 to the one using 2.14.5
  2. You do a fetchAuthSession before you do your operation
  3. Add the logging plugin as the first plugin in your list : Amplify.addPlugin(AndroidLoggingPlugin(LogLevel.VERBOSE))
  4. Paste the logs here

There could be an issue with migration and I would like to delve deeper to help you in this.

yaroslav-v commented 7 months ago

Thx for the answer.

I've added both fetchAuthSession and AndroidLoggingPlugin - the result is the same as previously. As far as I can see, fetchAuthSession receives credentials without any issues, nevertheless all uploads fail to finish.

Here is the log:

2023-12-16 13:13:48.904  8993-11324 APP          app.package  I  IdentityId: us-east-1:fa2f987b...

2023-12-16 13:13:49.005  8993-9152  amplify:aw...TransferDB app.package  I  update state for 138 to IN_PROGRESS

2023-12-16 13:13:22.814  8993-10940 amplify:aw...loadWorker app.package  E  SinglePartUploadWorker failed with exception: aws.sdk.kotlin.services.s3.model.S3Exception: The provided token has expired.
                                                                                            at aws.sdk.kotlin.services.s3.serde.PutObjectOperationDeserializerKt.b(SourceFile:159)
                                                                                            at aws.sdk.kotlin.services.s3.serde.PutObjectOperationDeserializerKt.a(SourceFile:1)
                                                                                            at aws.sdk.kotlin.services.s3.serde.PutObjectOperationDeserializerKt$throwPutObjectError$1.invokeSuspend(Unknown Source:10)
                                                                                            at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(SourceFile:12)
                                                                                            at kotlinx.coroutines.DispatchedTask.run(SourceFile:124)
                                                                                            at kotlinx.coroutines.scheduling.CoroutineScheduler.s(SourceFile:1)
                                                                                            at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.d(SourceFile:15)
                                                                                            at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.p(SourceFile:29)
                                                                                            at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(SourceFile:1)

2023-12-16 13:13:50.949  8993-10940 amplify:aw...TransferDB app.package  I  update state for 138 to FAILED

2023-12-16 13:13:22.822  8993-8993                          app.package  E  AWS capture failed
                                                                                        StorageException{message=Something went wrong with your AWS S3 Storage upload file operation, cause=java.lang.Exception: aws.sdk.kotlin.services.s3.model.S3Exception: The provided token has expired., recoverySuggestion=See attached exception for more information and suggestions}
                                                                                            at com.amplifyframework.storage.s3.operation.AWSS3StorageUploadFileOperation$UploadTransferListener.onError(SourceFile:33)
                                                                                            at com.amplifyframework.storage.s3.transfer.TransferStatusUpdater.updateOnError$lambda$11$lambda$10(SourceFile:11)
                                                                                            at com.amplifyframework.storage.s3.transfer.TransferStatusUpdater.c(SourceFile:1)
                                                                                            at com.amplifyframework.storage.s3.transfer.g.run(SourceFile:1)
                                                                                            at android.os.Handler.handleCallback(Handler.java:942)
                                                                                            at android.os.Handler.dispatchMessage(Handler.java:99)
                                                                                            at android.os.Looper.loopOnce(Looper.java:201)
                                                                                            at android.os.Looper.loop(Looper.java:288)
                                                                                            at android.app.ActivityThread.main(ActivityThread.java:7872)
                                                                                            at java.lang.reflect.Method.invoke(Native Method)
                                                                                            at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548)
                                                                                            at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:936)
                                                                                        Caused by: java.lang.Exception: aws.sdk.kotlin.services.s3.model.S3Exception: The provided token has expired.
                                                                                            at com.amplifyframework.storage.s3.transfer.worker.BaseTransferWorker.doWork$suspendImpl(SourceFile:386)
                                                                                            at com.amplifyframework.storage.s3.transfer.worker.BaseTransferWorker$doWork$1.invokeSuspend(Unknown Source:11)
                                                                                            at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(SourceFile:12)
                                                                                            at kotlinx.coroutines.UndispatchedCoroutine.b1(SourceFile:60)
                                                                                            at kotlinx.coroutines.AbstractCoroutine.resumeWith(SourceFile:16)
                                                                                            at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(SourceFile:48)
                                                                                            at kotlinx.coroutines.UndispatchedCoroutine.b1(SourceFile:60)
                                                                                            at kotlinx.coroutines.AbstractCoroutine.resumeWith(SourceFile:16)
                                                                                            at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(SourceFile:48)
                                                                                            at kotlinx.coroutines.DispatchedTask.run(SourceFile:124)
                                                                                            at kotlinx.coroutines.scheduling.CoroutineScheduler.s(SourceFile:1)
                                                                                            at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.d(SourceFile:15)
                                                                                            at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.p(SourceFile:29)
                                                                                            at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(SourceFile:1)
                                                                                        Caused by: aws.sdk.kotlin.services.s3.model.S3Exception: The provided token has expired.
                                                                                            at aws.sdk.kotlin.services.s3.serde.PutObjectOperationDeserializerKt.b(SourceFile:159)
                                                                                            at aws.sdk.kotlin.services.s3.serde.PutObjectOperationDeserializerKt.a(SourceFile:1)
                                                                                            at aws.sdk.kotlin.services.s3.serde.PutObjectOperationDeserializerKt$throwPutObjectError$1.invokeSuspend(Unknown Source:10)
                                                                                            at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(SourceFile:12)
                                                                                            at kotlinx.coroutines.DispatchedTask.run(SourceFile:124) 
                                                                                            at kotlinx.coroutines.scheduling.CoroutineScheduler.s(SourceFile:1) 
                                                                                            at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.d(SourceFile:15) 
                                                                                            at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.p(SourceFile:29) 
                                                                                            at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(SourceFile:1) 

023-12-16 13:13:50.965  8993-9137  WM-WorkerWrapper        app.package  I  Worker result FAILURE for Work [ id=41e4ff8f-61d9-4172-997c-48aed81831b0, tags={ com.amplifyframework.storage.s3.transfer.worker.RouterWorker, awsS3StoragePlugin, 138, UPLOAD } ]
gpanshu commented 7 months ago

I am going to locally try to reproduce this and get back to you.

gpanshu commented 7 months ago

One question I had forgotten to ask before: is your device tracking enabled in your user pool ?

yaroslav-v commented 7 months ago

Hmm... I'm not sure about this. As far as I know, this setting should be set by default.

gpanshu commented 7 months ago

Gotcha. By default device tracking is not enabled but whenever the backend was configured if it was selected then it will be enabled. What happens in your upgrade if you sign out with forcesignout and then login ? Is the problem resolved ?

yaroslav-v commented 7 months ago

Well, we're using guest accounts at this stage, as I wrote earlier. I'm not sure that it's possible to do "forcesignout and then login" in such case. If it isn't correct, give me more details, please.

You can see the code snippet in the original message.

yaroslav-v commented 7 months ago

Btw I want to bring your attention to the fact that the error itself is caused either by Caused by aws.sdk.kotlin.services.s3.model.S3Exception The provided token has expired.

or by Caused by aws.sdk.kotlin.services.s3.model.S3Exception The provided token is malformed or otherwise invalid.

I can't reproduce the second cause, but I see it in our Firebase logs in quite big numbers. Probably, it's not just expiration issue rather something with the token handling.

gpanshu commented 7 months ago

Yes you can do a forcesignout and then a fetchauthsession which will get you fresh guest credentials. The force signout will clean out all existing tokens.

yaroslav-v commented 7 months ago

I couldn't find forcesignout, so I've used normal Amplify.Auth.signOut method. Looks like it works and locally I can't reproduce the issue anymore.

We'll publish an update for the app soon. I'll be able to give more details after that.

P.S. I'm not sure if it's related or not, but I've noticed this warning message in the logs after updating the app:

AndroidKeysetManager    app.package  W  keyset not found, will generate a new one
                                                                                        java.io.FileNotFoundException: can't read keyset; the pref value __androidx_security_crypto_encrypted_prefs_value_keyset__ does not exist
                                                                                            at com.google.crypto.tink.integration.android.SharedPrefKeysetReader.b(SourceFile:33)
                                                                                            at com.google.crypto.tink.integration.android.SharedPrefKeysetReader.a(SourceFile:1)
                                                                                            at com.google.crypto.tink.KeysetHandle.j(SourceFile:1)
                                                                                            at com.google.crypto.tink.integration.android.AndroidKeysetManager$Builder.e(SourceFile:7)
                                                                                            at com.google.crypto.tink.integration.android.AndroidKeysetManager$Builder.f(SourceFile:1)
                                                                                            at com.google.crypto.tink.integration.android.AndroidKeysetManager$Builder.d(SourceFile:12)
                                                                                            at androidx.security.crypto.EncryptedSharedPreferences.a(SourceFile:93)
                                                                                            at com.amplifyframework.core.store.EncryptedKeyValueRepository$sharedPreferences$2.invoke(SourceFile:7)
                                                                                            at com.amplifyframework.core.store.EncryptedKeyValueRepository$sharedPreferences$2.invoke(SourceFile:1)
                                                                                            at kotlin.SynchronizedLazyImpl.getValue(SourceFile:21)
                                                                                            at com.amplifyframework.core.store.EncryptedKeyValueRepository.getSharedPreferences$com_amplifyframework_core_release(SourceFile:3)
                                                                                            at com.amplifyframework.core.store.EncryptedKeyValueRepository$editor$2.invoke(SourceFile:2)
                                                                                            at com.amplifyframework.core.store.EncryptedKeyValueRepository$editor$2.invoke(SourceFile:1)
                                                                                            at kotlin.SynchronizedLazyImpl.getValue(SourceFile:21)
                                                                                            at com.amplifyframework.core.store.EncryptedKeyValueRepository.getEditor$com_amplifyframework_core_release(SourceFile:3)
                                                                                            at com.amplifyframework.core.store.EncryptedKeyValueRepository.put(SourceFile:6)
                                                                                            at com.amplifyframework.auth.cognito.data.AWSCognitoAuthCredentialStore.saveCredential(SourceFile:18)
                                                                                            at com.amplifyframework.auth.cognito.actions.CredentialStoreCognitoActions$migrateLegacyCredentialStoreAction$$inlined$invoke$1.execute(SourceFile:58)
                                                                                            at com.amplifyframework.statemachine.ConcurrentEffectExecutor$execute$1$1.invokeSuspend(SourceFile:35)
                                                                                            at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(SourceFile:12)
                                                                                            at kotlinx.coroutines.DispatchedTask.run(SourceFile:124)
                                                                                            at kotlinx.coroutines.scheduling.CoroutineScheduler.s(SourceFile:1)
                                                                                            at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.d(SourceFile:15)
                                                                                            at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.p(SourceFile:29)
                                                                                            at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(SourceFile:1)
github-actions[bot] commented 7 months ago

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

tjleing commented 7 months ago

I saw @gpanshu closed this issue, but by the way @yaroslav-v the warning you mentioned is expected as we create an EncryptedSharedPreferences instance. If the issue persists please create a new issue.

yaroslav-v commented 5 months ago

FYI I've been tracking the state of this issue for a while in Firebase and see than the number of reports has fallen significantly after the fix was applied in the project. However, there are still some reports appear on a daily basis (using the latest version of the library 2.14.10).

I just want to point out that according to the docs AWS credentials for guest accounts should be refreshed automatically and the issue didn't exist for AWS Amplify v1 🤷‍♂️.

There's the mentioned doc https://docs.amplify.aws/android/sdk/auth/ - "AWS Credentials ... For Guest scenarios they will be automatically refreshed."