search

aws-amplify / amplify-flutter

A declarative library with an easy-to-use interface for building Flutter applications on AWS.
https://docs.amplify.aws
Apache License 2.0
1.31k stars 243 forks source link

Android-only DataStore Sync failure of a simple model with Dev Preview "1.0.0-next.2" library #2580

Closed vgribok closed 11 months ago

vgribok commented 1 year ago

Description

It's likely to be an Android platform specific issues, but as I have little knowledge of that and encountered the problem in Flutter, filling it here. Please feel free to move.

Here's the model:

type UserProfile @model
@auth(rules: [
    { allow: groups, groups: ["Admins"] } 
    { allow: owner, ownerField: "personId" }
])
{
    personId: String! @primaryKey
    firstName: String
    lastName: String!
}

The CLI didn't complain about the schema and it syncs fine on iOS but sync fails on Android with

E/amplify:aws-datastore( 8167): Caused by: DataStoreException{message=Error in saving the model: UserProfile[primaryKey =9fbc3582-7589-4f7e-bc69-4e68ab2b970b], cause=java.lang.NullPointerException: Attempt to invoke virtual method 'java.lang.String com.amplifyframework.datastore.storage.sqlite.adapter.SQLiteColumn.getName()' on a null object reference, recoverySuggestion=See attached exception for details.} E/amplify:aws-datastore( 8167): at com.amplifyframework.datastore.storage.sqlite.SQLiteStorageAdapter.lambda$save$3$SQLiteStorageAdapter(SQLiteStorageAdapter.java:382) E/amplify:aws-datastore( 8167): at com.amplifyframework.datastore.storage.sqlite.-$$Lambda$SQLiteStorageAdapter$0Q4t2Se5oVsIb4pxkwWPcTH3ADs.run(Unknown Source:12) E/amplify:aws-datastore( 8167): at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:462) E/amplify:aws-datastore( 8167): at java.util.concurrent.FutureTask.run(FutureTask.java:266) E/amplify:aws-datastore( 8167): ... 3 more E/amplify:aws-datastore( 8167): Caused by: java.lang.NullPointerException: Attempt to invoke virtual method 'java.lang.String com.amplifyframework.datastore.storage.sqlite.adapter.SQLiteColumn.getName()' on a null object reference E/amplify:aws-datastore( 8167): at com.amplifyframework.datastore.storage.sqlite.SqlQueryProcessor.modelExists(SqlQueryProcessor.java:95) E/amplify:aws-datastore( 8167): at com.amplifyframework.datastore.storage.sqlite.SQLiteStorageAdapter.lambda$save$3$SQLiteStorageAdapter(SQLiteStorageAdapter.java:335) E/amplify:aws-datastore( 8167): ... 6 more W/amplify:aws-datastore( 8167): API sync failed - transitioning to LOCAL_ONLY.

Ran it against Android API 31 and 33 with same outcome.

Feel free to DM me for the full stack trace and/or .graphql schema.

Categories

Steps to Reproduce

No response

Screenshots

No response

Platforms

Android Device/Emulator API Level

API 31

Environment

Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.3.3, on macOS 13.1 22C65 darwin-arm, locale en)
[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 14.0.1)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2021.3)
[✓] VS Code (version 1.70.2)
[✓] Connected device (4 available)
[✓] HTTP Host Availability

• No issues found!

Dependencies

❯ flutter pub deps --no-dev --style=compact
Dart SDK 2.18.2
Flutter SDK 3.3.3
redacted 1.0.0+1

dependencies:
- amplify_api 1.0.0-next.2+1 [amplify_api_android amplify_api_ios amplify_core amplify_flutter async aws_common collection connectivity_plus flutter json_annotation meta plugin_platform_interface stream_transform web_socket_channel]
- amplify_auth_cognito 1.0.0-next.2+1 [amplify_auth_cognito_android amplify_auth_cognito_dart amplify_auth_cognito_ios amplify_core amplify_flutter amplify_secure_storage async flutter flutter_web_plugins meta path plugin_platform_interface]
- amplify_authenticator 1.0.0-next.1+3 [amplify_auth_cognito amplify_core amplify_flutter async aws_common collection flutter flutter_localizations intl meta smithy stream_transform]
- amplify_datastore 1.0.0-next.2 [flutter amplify_datastore_plugin_interface amplify_core plugin_platform_interface meta collection async]
- amplify_flutter 1.0.0-next.2 [amplify_core amplify_datastore_plugin_interface amplify_flutter_android amplify_flutter_ios amplify_secure_storage aws_common collection flutter meta plugin_platform_interface]
- amplify_storage_s3 1.0.0-next.2 [amplify_core amplify_db_common amplify_storage_s3_dart aws_common flutter meta path_provider]
- async 2.9.0 [collection meta]
- connectivity_plus 3.0.2 [flutter flutter_web_plugins connectivity_plus_platform_interface js meta nm]
- cupertino_icons 1.0.5
- flutter 0.0.0 [characters collection material_color_utilities meta vector_math sky_engine]
- flutter_form_builder 7.7.0 [flutter intl collection]
- flutter_localizations 0.0.0 [flutter intl characters clock collection material_color_utilities meta path vector_math]
- flutter_native_splash 2.2.16 [args flutter flutter_web_plugins js html image meta path universal_io xml yaml]
- flutter_nav2_oop 1.0.0+1 [flutter cupertino_icons flutter_riverpod google_fonts riverpod_restorable]
- flutter_riverpod 2.1.3 [collection flutter meta riverpod state_notifier]
- font_awesome_flutter 10.3.0 [flutter]
- form_builder_validators 8.4.0 [flutter flutter_localizations intl]
- geolocator 9.0.2 [flutter geolocator_platform_interface geolocator_android geolocator_apple geolocator_web geolocator_windows]
- image_picker 0.8.6 [flutter image_picker_android image_picker_for_web image_picker_ios image_picker_platform_interface]
- intl 0.17.0 [clock path]
- native_exif 0.4.0 [flutter intl]
- path 1.8.2
- path_provider 2.0.11 [flutter path_provider_android path_provider_ios path_provider_linux path_provider_macos path_provider_platform_interface path_provider_windows]
- riverpod_restorable 2.0.0 [flutter riverpod flutter_riverpod]

transitive dependencies:
- amplify_api_android 1.0.0-next.2 [flutter]
- amplify_api_ios 1.0.0-next.2 [amplify_core flutter]
- amplify_auth_cognito_android 1.0.0-next.2 [flutter]
- amplify_auth_cognito_dart 0.3.1 [amplify_core amplify_secure_storage_dart async aws_common aws_signature_v4 built_collection built_value collection convert crypto fixnum http intl js json_annotation meta oauth2 path smithy smithy_aws stream_transform uuid worker_bee]
- amplify_auth_cognito_ios 1.0.0-next.2 [amplify_core flutter]
- amplify_core 1.0.0-next.2+1 [async aws_common aws_signature_v4 collection intl json_annotation logging meta retry uuid]
- amplify_datastore_plugin_interface 1.0.0-next.2 [amplify_core collection flutter meta]
- amplify_db_common 0.1.2 [amplify_db_common_dart drift flutter path path_provider]
- amplify_db_common_dart 0.1.3 [amplify_core async aws_common drift meta path sqlite3]
- amplify_flutter_android 1.0.0-next.2 [flutter]
- amplify_flutter_ios 1.0.0-next.2 [amplify_core flutter]
- amplify_secure_storage 0.1.4+1 [amplify_secure_storage_dart async file flutter meta path path_provider]
- amplify_secure_storage_dart 0.1.4+1 [async aws_common built_collection built_value ffi file js meta path win32 worker_bee]
- amplify_storage_s3_dart 0.1.4 [amplify_core amplify_db_common_dart async aws_common aws_signature_v4 built_collection built_value drift fixnum meta path smithy smithy_aws]
- archive 3.3.0 [crypto path]
- args 2.3.1
- aws_common 0.3.5+1 [async built_collection built_value collection http2 js json_annotation logging meta mime os_detect path stream_transform uuid]
- aws_signature_v4 0.3.1+1 [async aws_common collection convert crypto json_annotation meta path]
- built_collection 5.1.1
- built_value 8.4.3 [built_collection collection fixnum meta]
- characters 1.2.1
- clock 1.1.1
- collection 1.16.0
- connectivity_plus_platform_interface 1.2.3 [flutter meta plugin_platform_interface]
- convert 3.1.1 [typed_data]
- crclib 3.0.0 [meta tuple]
- cross_file 0.3.3+2 [js meta]
- crypto 3.0.2 [typed_data]
- csslib 0.17.2 [source_span]
- dbus 0.7.8 [args ffi meta xml]
- drift 2.3.0 [async convert collection js meta stream_channel sqlite3]
- ffi 2.0.1
- file 6.1.2 [meta path]
- fixnum 1.0.1
- flutter_plugin_android_lifecycle 2.0.7 [flutter]
- flutter_web_plugins 0.0.0 [flutter js characters collection material_color_utilities meta vector_math]
- geolocator_android 4.1.4 [flutter geolocator_platform_interface]
- geolocator_apple 2.2.3 [flutter geolocator_platform_interface]
- geolocator_platform_interface 4.0.7 [flutter plugin_platform_interface vector_math meta]
- geolocator_web 2.1.6 [flutter flutter_web_plugins geolocator_platform_interface]
- geolocator_windows 0.1.1 [flutter geolocator_platform_interface]
- google_fonts 3.0.1 [flutter http path_provider crypto]
- html 0.15.1 [csslib source_span]
- http 0.13.5 [async http_parser meta path]
- http2 2.0.1
- http_parser 4.0.2 [collection source_span string_scanner typed_data]
- image 3.3.0 [archive meta xml]
- image_picker_android 0.8.5+4 [flutter flutter_plugin_android_lifecycle image_picker_platform_interface]
- image_picker_for_web 2.1.10 [flutter flutter_web_plugins image_picker_platform_interface]
- image_picker_ios 0.8.6+5 [flutter image_picker_platform_interface]
- image_picker_platform_interface 2.6.2 [cross_file flutter http plugin_platform_interface]
- js 0.6.4
- json_annotation 4.7.0 [meta]
- logging 1.1.0
- material_color_utilities 0.1.5
- meta 1.8.0
- mime 1.0.3
- nm 0.5.0 [dbus]
- oauth2 2.0.1 [collection crypto http http_parser]
- os_detect 2.0.1
- path_provider_android 2.0.22 [flutter path_provider_platform_interface]
- path_provider_ios 2.0.11 [flutter path_provider_platform_interface]
- path_provider_linux 2.1.7 [ffi flutter path path_provider_platform_interface xdg_directories]
- path_provider_macos 2.0.6 [flutter path_provider_platform_interface]
- path_provider_platform_interface 2.0.5 [flutter platform plugin_platform_interface]
- path_provider_windows 2.1.3 [ffi flutter path path_provider_platform_interface win32]
- petitparser 5.1.0 [meta]
- platform 3.1.0
- plugin_platform_interface 2.1.3 [meta]
- process 4.2.4 [file path platform]
- retry 3.1.0
- riverpod 2.1.3 [collection meta stack_trace state_notifier]
- shelf 1.4.0 [async collection http_parser path stack_trace stream_channel]
- sky_engine 0.0.99
- smithy 0.3.2 [async aws_common built_collection built_value collection convert crypto fixnum http_parser intl json_annotation meta path retry shelf typed_data xml]
- smithy_aws 0.3.2 [aws_common aws_signature_v4 built_collection built_value collection convert crclib crypto intl json_annotation meta path smithy xml]
- source_span 1.9.0 [collection path term_glyph]
- sqlite3 1.9.1 [collection ffi js meta path]
- stack_trace 1.10.0 [path]
- state_notifier 0.7.2+1 [meta]
- stream_channel 2.1.0 [async]
- stream_transform 2.1.0
- string_scanner 1.1.1 [source_span]
- term_glyph 1.2.1
- tuple 2.0.1
- typed_data 1.3.1 [collection]
- universal_io 2.0.4 [collection crypto meta typed_data]
- uuid 3.0.7 [crypto]
- vector_math 2.1.2
- web_socket_channel 2.3.0 [async crypto stream_channel]
- win32 3.1.3 [ffi]
- worker_bee 0.1.3+1 [async aws_common built_collection built_value collection js meta path stack_trace stream_channel stream_transform]
- xdg_directories 0.2.0+3 [meta path process]
- xml 6.1.0 [collection meta petitparser]
- yaml 3.1.1 [collection source_span string_scanner]

Device

Android Emulator of Pixel 3a on MacOS M1

OS

Android 33

Deployment Method

Amplify CLI

CLI Version

10.6.1

Additional Context

No response

Amplify Config

Instructions unclear

HuiSF commented 1 year ago

Hi @vgribok with what operation on the model would trigger this exception for you?

vgribok commented 1 year ago

Hi @vgribok with what operation on the model would trigger this exception for you?

I was running an observed query for a different model:

StreamProvider<List<T>>((ref) {
     final Stream<QuerySnapshot<T>> snapshotStream = Amplify.DataStore.observeQuery(
           SomeOtherModelType, where: where, sortBy: sortBy, throttleOptions: throttleOptions
     );
     return snapshotStream.map((snapshot) => snapshot.items);
})
vgribok commented 1 year ago

@HuiSF I went on a wild goose chase trying to come up with more data for this issue. Here are the findings.

I upgraded to "1.0.0-next.3".

I changed the schema so that the models would have id fields for ownerId:

type UserProfile @model
@auth(rules: [
    { allow: groups, groups: ["GsAdmins"] } # Unrestricted access to members of the "admin" user pool group
    { allow: owner, ownerField: "id" }
])
{
    id: String! @primaryKey
    firstName: String
    lastName: String!
}

That required to run amplify api remove because not even amplify api rebuild --allow-destructive-graphql-schema-updates could deal with the change. Then I restored project changes by doing git revert followed by amplify init and amplify api push. Not a smooth experience, but I got through it.

After I regenerated the models and ran the app, on iOS it was still running fine - the sync back and forth, but on Android the sync is broken seemingly in more ways than one. This time the inability to sync was due to an authorization issue of some kind, which is not observed on iOS with exacly the same flow of logging in and out with Cogntio AutheticatedView:

W/amplify:aws-datastore(15348): API sync failed - transitioning to LOCAL_ONLY.
W/amplify:aws-datastore(15348): DataStoreException{message=Initial sync during DataStore initialization failed., cause=io.reactivex.rxjava3.exceptions.CompositeException: 2 exceptions occurred. , recoverySuggestion=There is a possibility that there is a bug if this error persists. Please take a look at
W/amplify:aws-datastore(15348): https://github.com/aws-amplify/amplify-android/issues to see if there are any existing issues that
W/amplify:aws-datastore(15348): match your scenario, and file an issue with the details of the bug if there isn't.}
W/amplify:aws-datastore(15348):     at com.amplifyframework.datastore.syncengine.Orchestrator.lambda$startApiSync$3$Orchestrator(Orchestrator.java:324)
W/amplify:aws-datastore(15348):     at com.amplifyframework.datastore.syncengine.-$$Lambda$Orchestrator$PVk58tU0K8ndPJYnH_tRmf4RGwE.subscribe(Unknown Source:2)
W/amplify:aws-datastore(15348):     
...
W/amplify:aws-datastore(15348): Caused by: io.reactivex.rxjava3.exceptions.CompositeException$ExceptionOverview: Multiple exceptions (2)
W/amplify:aws-datastore(15348): |-- com.amplifyframework.datastore.DataStoreException: Failure performing sync query to AppSync: [GraphQLResponse.Error{message='Not Authorized to access syncDestinations on type ModelDestinationConnection', locations='[GraphQLLocation{line='2', column='3'}]', path='[GraphQLPathSegment{value='syncDestinations'}]', extensions='{errorInfo=null, data=null, errorType=Unauthorized}'}]
W/amplify:aws-datastore(15348):     at com.amplifyframework.datastore.appsync.AppSyncClient.lambda$sync$0(AppSyncClient.java:116)
W/amplify:aws-datastore(15348): |-- java.lang.RuntimeException: Methods marked with @UiThread must be executed on the main thread. Current thread: RxCachedThreadScheduler-151
W/amplify:aws-datastore(15348):     at io.flutter.embedding.engine.FlutterJNI.ensureRunningOnMainThread(FlutterJNI.java:1415)
I/amplify:aws-datastore(15348): Orchestrator transitioning from SYNC_VIA_API to LOCAL_ONLY
I/amplify:aws-datastore(15348): Setting currentState to LOCAL_ONLY
I/amplify:aws-datastore(15348): Stopping subscription processor.

But what was really surprising that after reporting all of the above, it actually down-synced a few records from the cloud:

I/amplify:aws-datastore(15348): Setting currentState to LOCAL_ONLY
I/amplify:aws-datastore(15348): Stopping subscription processor.
I/amplify:aws-datastore(15348): Stopped subscription processor.
W/tstride.logbook(15348): Long monitor contention with owner pool-4-thread-169 (17061) at void com.amplifyframework.api.aws.SubscriptionEndpoint.releaseSubscription(java.lang.String)(SubscriptionEndpoint.java:291) waiters=5 in void 
... lots of these noisy log records removed ^^ ....
com.amplifyframework.api.aws.SubscriptionEndpoint.releaseSubscription(java.lang.String) for 1.584s
D/EGL_emulation(15348): app_time_stats: avg=575.16ms min=1.41ms max=10531.65ms count=19
I/flutter (15348): DataStore query returned 2 items of type UserCertification

After I uninstalled the app on Android (emulator) and ran it again, I could no longer downsync anything - the sync remains dead. (Update: down-sync works sometimes, it's not fully dead like up-sync.)

To summarize, Android does not up-sync because of an authorization issue that iOS does not have, and despite of the Auth issue, it down-syncs sometimes.

vgribok commented 1 year ago

The root cause is that DataStore sync fails when encountering models for which the current user is unauthorized. I am not sure why this is an Andoird-only failure, though. It should either fail on all platforms, or succeed on all platforms. Also, if a user is unauthorized for a model, the sync should just skip syncing the model, not switch to the local only mode.

Jordan-Nelson commented 1 year ago

@vgribok - This issue fell off our radar. Apologies about that.

I would like to try to reproduce this to see if it is still an issue, but want to confirm that I have the correct reproduction steps.

Can you confirm these reproduction steps are correct?

  1. Using the schema below, save records and sync them to the cloud
  2. Sign in with a user that is not authorized to read the data (not an admin, not an owner)
  3. Attempt to sync data to the device
  4. Observe that initial sync fails.
type UserProfile @model
@auth(rules: [
    { allow: groups, groups: ["GsAdmins"] } # Unrestricted access to members of the "admin" user pool group
    { allow: owner, ownerField: "id" }
])
{
    id: String! @primaryKey
    firstName: String
    lastName: String!
}
Jordan-Nelson commented 1 year ago

@vgribok - Let us know if you recall the reproduction steps for this issue. Otherwise we will close this one out. Thanks.

vgribok commented 1 year ago

@Jordan-Nelson, yes the steps are correct. Please note that on iOS there was not problem, and the failure occurred only on Android.

Jordan-Nelson commented 11 months ago

@vgribok I am unable to reproduce this. Below are the steps I took and the app/schema that I used. If you are able to reproduce this with the latest version of Amplify, please let me know.

Steps:

  1. Sign up with user 1, save an item with the id equal to the user sub
  2. Repeat for user 2 and user 3
  3. Clear the local DB (DataStore.clear() and sign out), sign in with user 1 and see that data sync correctly
  4. Repeat for user 2 and user 3

Schema:

```graphql type UserProfile @model @auth(rules: [ { allow: groups, groups: ["GsAdmins"] } # Unrestricted access to members of the "admin" user pool group { allow: owner, ownerField: "id" } ]) { id: String! @primaryKey firstName: String lastName: String! } ```

App:

```dart import 'package:amplify_api/amplify_api.dart'; import 'package:amplify_auth_cognito/amplify_auth_cognito.dart'; import 'package:amplify_authenticator/amplify_authenticator.dart'; import 'package:amplify_datastore/amplify_datastore.dart'; import 'package:amplify_flutter/amplify_flutter.dart' hide UserProfile; import 'package:flutter/material.dart'; import 'package:issue_2580/models/ModelProvider.dart'; import 'amplifyconfiguration.dart'; void main() { runApp(const MyApp()); } class MyApp extends StatefulWidget { const MyApp({super.key}); @override State createState() => _MyAppState(); } class _MyAppState extends State { @override void initState() { super.initState(); _configureAmplify(); } void _configureAmplify() async { try { await Amplify.addPlugin(AmplifyAuthCognito()); await Amplify.addPlugin(AmplifyAPI()); await Amplify.addPlugin(AmplifyDataStore( modelProvider: ModelProvider.instance, )); await Amplify.configure(amplifyconfig); safePrint('Successfully configured'); } on Exception catch (e) { safePrint('Error configuring Amplify: $e'); } } @override Widget build(BuildContext context) { return Authenticator( child: MaterialApp( builder: Authenticator.builder(), home: const HomeWidget(), ), ); } } class HomeWidget extends StatefulWidget { const HomeWidget({ super.key, }); @override State createState() => _HomeWidgetState(); } class _HomeWidgetState extends State { List users = []; @override void initState() { Amplify.DataStore.observeQuery(UserProfile.classType).listen((event) { setState(() { users = event.items; }); }); super.initState(); } @override Widget build(BuildContext context) { return Scaffold( appBar: AppBar( actions: [ IconButton( onPressed: () async { await Amplify.DataStore.clear(); }, icon: const Icon(Icons.delete)) ], ), body: Center( child: Column( children: [ const Text('You are logged in!'), const SignOutButton(), ElevatedButton( onPressed: () async { final attr = await Amplify.Auth.fetchUserAttributes(); final email = attr.firstWhere( (element) => element.userAttributeKey == AuthUserAttributeKey.email, ); final sub = attr.firstWhere( (element) => element.userAttributeKey == AuthUserAttributeKey.sub, ); await Amplify.DataStore.save( UserProfile(id: sub.value, lastName: email.value), ); }, child: const Text('Save'), ), for (final user in users) ListTile(title: Text(user.lastName)), ], ), ), ); } } ```
haverchuck commented 11 months ago

Closing due to inactivity.