search

aws-amplify / amplify-flutter

A declarative library with an easy-to-use interface for building Flutter applications on AWS.
https://docs.amplify.aws
Apache License 2.0
1.32k stars 246 forks source link

Datastore save operation is incomplete when series of writes happens to model with @primaryKey #2519

Closed skim037 closed 6 months ago

skim037 commented 1 year ago

Description

I noticed that this issue was closed. I tried again but the issue is still there.

I will try to provide clear repro steps.

model

Make sure to use @primaryKey annotation. 

type TestTableOne @model @auth(rules: [{allow: public}]) {
  id: ID!
  profile_id: ID! @primaryKey
  count: Int
} 

type TestTableTwo @model @auth(rules: [{allow: public}]) {
  id: ID!
  profile_id: ID! @primaryKey
  count: Int
} 

type TestTableThree @model @auth(rules: [{allow: public}]) {
  id: ID!
  profile_id: ID! @primaryKey
  count: Int
}

Code to save model

Make sure to use same profileId for all three models. 

final profileId = UUID.getUUID();
final testTableOne = TestTableOne(profile_id: profileId, count: 100);
await ddbRepo.saveTestOne(testTableOne);
final testTableTwo = TestTableTwo(profile_id: profileId, count: 101);
await ddbRepo.saveTestTwo(testTableTwo);
final testTableThree = TestTableThree(profile_id: profileId, count: 102);
await ddbRepo.saveTestThree(testTableThree);

Repository class

  Future<void> saveTestOne(TestTableOne testTableOne) async {
    await Amplify.DataStore.save(testTableOne);
  }

  Future<void> saveTestTwo(TestTableTwo testTableTwo) async {
    await Amplify.DataStore.save(testTableTwo);
  }

  Future<void> saveTestThree(TestTableThree testTableThree) async {
    await Amplify.DataStore.save(testTableThree);
  }

Result I expect

A record in TestTableOne with profileId and count 100 A record in TestTableTwo with profileId and count 101 (Same profileId as in TestTableOne) A record in TestTableThree with profileId and count 102 (Same profileId as in TestTableOne)

Result I see

A record in TestTableOne with profileId and count 100 A record in TestTableTwo with the same profileId and count 101 No record in TestTableThree

Interesting observations

Categories

Steps to Reproduce

No response

Screenshots

No response

Platforms

Android Device/Emulator API Level

No response

Environment

Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.3.10, on macOS 13.0 22A380 darwin-arm, locale en-US)
[✓] Android toolchain - develop for Android devices (Android SDK version 32.1.0-rc1)
[✓] Xcode - develop for iOS and macOS (Xcode 14.2)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2021.3)
[✓] VS Code (version 1.74.0)
[✓] Connected device (4 available)
[✓] HTTP Host Availability

• No issues found!

Dependencies

Dart SDK 2.18.6
Flutter SDK 3.3.10
chummy 1.0.0+1

dependencies:
- amplify_analytics_pinpoint 0.6.10 [amplify_analytics_pinpoint_android amplify_analytics_pinpoint_ios amplify_core aws_common flutter meta]
- amplify_api 0.6.10 [amplify_api_android amplify_api_ios amplify_core amplify_flutter aws_common collection flutter meta plugin_platform_interface]
- amplify_auth_cognito 0.6.10 [amplify_auth_cognito_android amplify_auth_cognito_ios amplify_core aws_common collection flutter meta plugin_platform_interface]
- amplify_authenticator 0.2.4 [amplify_auth_cognito amplify_core amplify_flutter async aws_common collection flutter flutter_localizations intl stream_transform]
- amplify_datastore 0.6.10 [flutter amplify_datastore_plugin_interface amplify_core plugin_platform_interface meta collection async]
- amplify_flutter 0.6.10 [amplify_core amplify_datastore_plugin_interface amplify_flutter_android amplify_flutter_ios aws_common collection flutter meta plugin_platform_interface]
- amplify_storage_s3 0.6.10 [amplify_storage_s3_android amplify_storage_s3_ios amplify_core aws_common flutter meta plugin_platform_interface path_provider path]
- cached_network_image 3.2.3 [flutter flutter_cache_manager octo_image cached_network_image_platform_interface cached_network_image_web]
- connectivity_plus 3.0.2 [flutter flutter_web_plugins connectivity_plus_platform_interface js meta nm]
- cupertino_icons 1.0.5
- dotted_border 2.0.0+2 [flutter path_drawing]
- dropdown_button2 1.8.5 [flutter]
- email_validator 2.1.17
- firebase_core 2.3.0 [firebase_core_platform_interface firebase_core_web flutter meta]
- firebase_messaging 14.1.1 [firebase_core firebase_core_platform_interface firebase_messaging_platform_interface firebase_messaging_web flutter meta]
- flutter 0.0.0 [characters collection material_color_utilities meta vector_math sky_engine]
- flutter_app_badger 1.5.0 [flutter]
- flutter_bloc 8.1.1 [flutter bloc provider]
- flutter_cache_manager 3.3.0 [clock collection file flutter http path path_provider pedantic rxdart sqflite uuid]
- flutter_image_compress 1.1.3 [flutter]
- flutter_local_notifications 12.0.4 [clock flutter flutter_local_notifications_linux flutter_local_notifications_platform_interface timezone]
- flutter_native_splash 2.2.15 [args flutter flutter_web_plugins js html image meta path universal_io xml yaml]
- flutter_slidable 2.0.0 [flutter]
- geocoding 2.0.5 [flutter geocoding_platform_interface]
- geolocator 9.0.2 [flutter geolocator_platform_interface geolocator_android geolocator_apple geolocator_web geolocator_windows]
- google_maps_flutter 2.2.1 [flutter google_maps_flutter_android google_maps_flutter_ios google_maps_flutter_platform_interface]
- google_mobile_ads 2.3.0 [meta flutter visibility_detector]
- google_place 0.4.7 [http]
- image_picker 0.8.6 [flutter image_picker_android image_picker_for_web image_picker_ios image_picker_platform_interface]
- intl 0.17.0 [clock path]
- material_design_icons_flutter 6.0.7096 [flutter]
- path_provider 2.0.11 [flutter path_provider_android path_provider_ios path_provider_linux path_provider_macos path_provider_platform_interface path_provider_windows]
- percent_indicator 4.2.2 [flutter]
- permission_handler 10.2.0 [flutter meta permission_handler_android permission_handler_apple permission_handler_windows permission_handler_platform_interface]
- quiver 3.1.0 [matcher]
- social_login_buttons 1.0.7 [flutter]
- url_launcher 6.1.7 [flutter url_launcher_android url_launcher_ios url_launcher_linux url_launcher_macos url_launcher_platform_interface url_launcher_web url_launcher_windows]

transitive dependencies:
- _flutterfire_internals 1.0.9 [cloud_firestore_platform_interface cloud_firestore_web collection firebase_core firebase_core_platform_interface flutter meta]
- amplify_analytics_pinpoint_android 0.6.10 [flutter]
- amplify_analytics_pinpoint_ios 0.6.10 [flutter]
- amplify_api_android 0.6.10 [flutter]
- amplify_api_ios 0.6.10 [amplify_core flutter]
- amplify_auth_cognito_android 0.6.10 [flutter]
- amplify_auth_cognito_ios 0.6.10 [amplify_core flutter]
- amplify_core 0.6.10 [aws_common collection flutter intl json_annotation meta plugin_platform_interface uuid]
- amplify_datastore_plugin_interface 0.6.10 [amplify_core collection flutter meta]
- amplify_flutter_android 0.6.10 [flutter]
- amplify_flutter_ios 0.6.10 [amplify_core flutter]
- amplify_storage_s3_android 0.6.10 [flutter]
- amplify_storage_s3_ios 0.6.10 [flutter]
- archive 3.3.4 [crypto path pointycastle]
- args 2.3.1
- async 2.9.0 [collection meta]
- aws_common 0.1.1 [async collection http meta stream_transform uuid]
- bloc 8.1.0 [meta]
- boolean_selector 2.1.0 [source_span string_scanner]
- cached_network_image_platform_interface 2.0.0 [flutter flutter_cache_manager]
- cached_network_image_web 1.0.2 [flutter flutter_cache_manager cached_network_image_platform_interface]
- characters 1.2.1
- clock 1.1.1
- cloud_firestore_platform_interface 5.9.0 [_flutterfire_internals collection firebase_core flutter meta plugin_platform_interface]
- cloud_firestore_web 3.1.0 [_flutterfire_internals cloud_firestore_platform_interface collection firebase_core firebase_core_web flutter flutter_web_plugins js]
- collection 1.16.0
- connectivity_plus_platform_interface 1.2.3 [flutter meta plugin_platform_interface]
- convert 3.1.1 [typed_data]
- cross_file 0.3.3+2 [js meta]
- crypto 3.0.2 [typed_data]
- csslib 0.17.2 [source_span]
- dbus 0.7.8 [args ffi meta xml]
- fake_async 1.3.1 [clock collection]
- ffi 2.0.1
- file 6.1.4 [meta path]
- firebase_core_platform_interface 4.5.2 [collection flutter flutter_test meta plugin_platform_interface]
- firebase_core_web 2.0.1 [firebase_core_platform_interface flutter flutter_web_plugins js meta]
- firebase_messaging_platform_interface 4.2.7 [_flutterfire_internals firebase_core flutter meta plugin_platform_interface]
- firebase_messaging_web 3.2.7 [_flutterfire_internals firebase_core firebase_core_web firebase_messaging_platform_interface flutter flutter_web_plugins js meta]
- flutter_blurhash 0.7.0 [flutter]
- flutter_local_notifications_linux 2.0.0 [flutter flutter_local_notifications_platform_interface dbus path xdg_directories]
- flutter_local_notifications_platform_interface 6.0.0 [flutter plugin_platform_interface]
- flutter_localizations 0.0.0 [flutter intl characters clock collection material_color_utilities meta path vector_math]
- flutter_plugin_android_lifecycle 2.0.7 [flutter]
- flutter_test 0.0.0 [flutter test_api path fake_async clock stack_trace vector_math async boolean_selector characters collection matcher material_color_utilities meta source_span stream_channel string_scanner term_glyph]
- flutter_web_plugins 0.0.0 [flutter js characters collection material_color_utilities meta vector_math]
- geocoding_platform_interface 2.0.1 [flutter meta plugin_platform_interface]
- geolocator_android 4.1.4 [flutter geolocator_platform_interface]
- geolocator_apple 2.2.2 [flutter geolocator_platform_interface]
- geolocator_platform_interface 4.0.6 [flutter plugin_platform_interface vector_math meta]
- geolocator_web 2.1.6 [flutter flutter_web_plugins geolocator_platform_interface]
- geolocator_windows 0.1.1 [flutter geolocator_platform_interface]
- google_maps_flutter_android 2.3.2 [flutter flutter_plugin_android_lifecycle google_maps_flutter_platform_interface stream_transform]
- google_maps_flutter_ios 2.1.12 [flutter google_maps_flutter_platform_interface stream_transform]
- google_maps_flutter_platform_interface 2.2.4 [collection flutter plugin_platform_interface stream_transform]
- html 0.15.1 [csslib source_span]
- http 0.13.5 [async http_parser meta path]
- http_parser 4.0.1 [collection source_span string_scanner typed_data]
- image 3.2.2 [archive meta xml]
- image_picker_android 0.8.5+3 [flutter flutter_plugin_android_lifecycle image_picker_platform_interface]
- image_picker_for_web 2.1.10 [flutter flutter_web_plugins image_picker_platform_interface]
- image_picker_ios 0.8.6+1 [flutter image_picker_platform_interface]
- image_picker_platform_interface 2.6.2 [cross_file flutter http plugin_platform_interface]
- js 0.6.4
- json_annotation 4.7.0 [meta]
- matcher 0.12.12 [stack_trace]
- material_color_utilities 0.1.5
- meta 1.8.0
- nested 1.0.0 [flutter]
- nm 0.5.0 [dbus]
- octo_image 1.0.2 [flutter flutter_blurhash]
- path 1.8.2
- path_drawing 1.0.1 [vector_math meta path_parsing flutter]
- path_parsing 1.0.1 [vector_math meta]
- path_provider_android 2.0.20 [flutter path_provider_platform_interface]
- path_provider_ios 2.0.11 [flutter path_provider_platform_interface]
- path_provider_linux 2.1.7 [ffi flutter path path_provider_platform_interface xdg_directories]
- path_provider_macos 2.0.6 [flutter path_provider_platform_interface]
- path_provider_platform_interface 2.0.5 [flutter platform plugin_platform_interface]
- path_provider_windows 2.1.3 [ffi flutter path path_provider_platform_interface win32]
- pedantic 1.11.1
- permission_handler_android 10.2.0 [flutter permission_handler_platform_interface]
- permission_handler_apple 9.0.7 [flutter permission_handler_platform_interface]
- permission_handler_platform_interface 3.9.0 [flutter meta plugin_platform_interface]
- permission_handler_windows 0.1.2 [flutter permission_handler_platform_interface]
- petitparser 5.1.0 [meta]
- platform 3.1.0
- plugin_platform_interface 2.1.3 [meta]
- pointycastle 3.6.2 [collection convert js]
- process 4.2.4 [file path platform]
- provider 6.0.3 [collection flutter nested]
- rxdart 0.27.5
- sky_engine 0.0.99
- source_span 1.9.0 [collection path term_glyph]
- sqflite 2.1.0+1 [flutter sqflite_common path]
- sqflite_common 2.3.0 [synchronized path meta]
- stack_trace 1.10.0 [path]
- stream_channel 2.1.0 [async]
- stream_transform 2.0.0
- string_scanner 1.1.1 [source_span]
- synchronized 3.0.0+3
- term_glyph 1.2.1
- test_api 0.4.12 [async boolean_selector collection meta source_span stack_trace stream_channel string_scanner term_glyph matcher]
- timezone 0.9.0 [path]
- typed_data 1.3.1 [collection]
- universal_io 2.0.4 [collection crypto meta typed_data]
- url_launcher_android 6.0.22 [flutter url_launcher_platform_interface]
- url_launcher_ios 6.0.17 [flutter url_launcher_platform_interface]
- url_launcher_linux 3.0.1 [flutter url_launcher_platform_interface]
- url_launcher_macos 3.0.1 [flutter url_launcher_platform_interface]
- url_launcher_platform_interface 2.1.1 [flutter plugin_platform_interface]
- url_launcher_web 2.0.13 [flutter flutter_web_plugins url_launcher_platform_interface]
- url_launcher_windows 3.0.1 [flutter url_launcher_platform_interface]
- uuid 3.0.6 [crypto]
- vector_math 2.1.2
- visibility_detector 0.3.3 [flutter]
- win32 3.0.1 [ffi]
- xdg_directories 0.2.0+2 [meta path process]
- xml 6.1.0 [collection meta petitparser]
- yaml 3.1.1 [collection source_span string_scanner]

Device

iPhone 12

OS

iOS 16.1.2

Deployment Method

Amplify CLI

CLI Version

10.5.2

Additional Context

No response

Amplify Config

{}

fjnoyp commented 1 year ago

Hi @skim037 thanks for reposting this issue. I've copied your model schemas and code and will test later tomorrow.

fjnoyp commented 1 year ago

Hi @skim037 I am able to reproduce this issue and get the following error:

Unhandled Exception: DataStoreException(message: Received a create mutation for an item that has already been created, recoverySuggestion: Review your app code and ensure you are not issuing incorrect DataStore.save() calls for the same model.

Many people are gone for holidays now and I'd like to double check with them if using the same custom primary key value for 3 different models is expected or recommended in your case and in general.

From what you are doing I imagine you want to use the same custom primary key to easily 'link' these 3 different model types together. Could you instead use 1 primary model that has a '@hasOne' relationship to the 3 different model types you need. So something like below. You can access the Parent model with the profileId and then use that to access the testTableOne/Two/Three objects. 

type Parent @model {
    id: ID!
    profile_id: ID! @primaryKey 
    testTableChild: TestTableOne @hasOne
    testTableChild2: TestTableTwo @hasOne
    testTableChild3: TestTableThree @hasOne
}

type TestTableOne @model @auth(rules: [{allow: public}]) {
  id: ID!
  profile_id: ID! @primaryKey
  count: Int
} 

type TestTableTwo @model @auth(rules: [{allow: public}]) {
  id: ID!
  profile_id: ID! @primaryKey
  count: Int
} 

type TestTableThree @model @auth(rules: [{allow: public}]) {
  id: ID!
  profile_id: ID! @primaryKey
  count: Int
}
skim037 commented 1 year ago

Hi @fjnoyp

I think the model change you propose would work if I use unique profile_id for all three models but that would defeat the purpose of using custom primary key in the first place, wouldn't it? Sorry if I didn't get your point properly.

Unless we fully understand what conditions would trigger such exception, it makes us very uncomfortable to use custom primary key at all.

HuiSF commented 1 year ago

Hi @skim037 thanks for all the details, I'm wondering how to you query by using the value of profile_id? I hope to understand your case better.

In addition to also checked that this issue is applicable to Android, have you seen the exact behavior in Android as well please?

@fjnoyp has confirm that the same exception appears in Android as well.

HuiSF commented 1 year ago

Dug into the implementation which throws this exception in amplify-swift it looks like the logic doesn't allow different model types having the same primary key value (not really custom primary key feature specific). If I remembered correctly, there was an effort in both amplify-swift and amplify-android to allow this use case. This might be a missing spot. I will follow up with amplify-swift and amplify-android maintainers to confirm and determine a solution.

Jordan-Nelson commented 1 year ago

@skim037 - I believe this issue has been addressed in the native iOS library. Were you seeing this issue in iOS only, or was it an issue for you on Android as well?

Equartey commented 1 year ago

@skim037 - I've confirmed this is working in amplify_datastore: ^1.4.0. I'm going to close this as resolved. Please reach out if you have any concerns.

Equartey commented 1 year ago

Apologies, this is fixed in iOS, but I'm seeing an exception being thrown on Android. Will work with the Android team to get it resolved.

Android exception: 

Sync failed: foreign key constraint violation: ModelWithMetadata{model=SerializedModel{id='1a696661-2d5a-47e7-b4c4-6d57fcfd63d8', serializedData={count=100, createdAt=2023-10-06T18:37:45.532Z, id=6447a0cf-1641-44de-921e-8939c7c611ca, profile_id=1a696661-2d5a-47e7-b4c4-6d57fcfd63d8, updatedAt=2023-10-06T18:37:45.532Z}, modelName=TestTableOne}, syncMetadata=ModelMetadata{id='TestTableOne|1a696661-2d5a-47e7-b4c4-6d57fcfd63d8', _deleted=null, _version=1, _lastChangedAt=Temporal.Timestamp{timestamp=1696617465563}}}
W/amplify:aws-datastore(17630): DataStoreException{message=Invalid SQL statement: INSERT INTO `TestTableOne` (`id`, `count`, `createdAt`, `profile_id`, `updatedAt`) VALUES (?, ?, ?, ?, ?), cause=android.database.sqlite.SQLiteConstraintException: UNIQUE constraint failed: TestTableOne.profile_id (code 1555 SQLITE_CONSTRAINT_PRIMARYKEY), recoverySuggestion=There is a possibility that there is a bug if this error persists. Please take a look at 
W/amplify:aws-datastore(17630): https://github.com/aws-amplify/amplify-android/issues to see if there are any existing issues that 
W/amplify:aws-datastore(17630): match your scenario, and file an issue with the details of the bug if there isn't.}
W/amplify:aws-datastore(17630):     at com.amplifyframework.datastore.storage.sqlite.SQLCommandProcessor.dataStoreException(SQLCommandProcessor.java:87)
W/amplify:aws-datastore(17630):     at com.amplifyframework.datastore.storage.sqlite.SQLCommandProcessor.execute(SQLCommandProcessor.java:82)
W/amplify:aws-datastore(17630):     at com.amplifyframework.datastore.storage.sqlite.SQLiteStorageAdapter.writeData(SQLiteStorageAdapter.java:769)
W/amplify:aws-datastore(17630):     at com.amplifyframework.datastore.storage.sqlite.SQLiteStorageAdapter.lambda$save$3$com-amplifyframework-datastore-storage-sqlite-SQLiteStorageAdapter(SQLiteStorageAdapter.java:366)
W/amplify:aws-datastore(17630):     at com.amplifyframework.datastore.storage.sqlite.SQLiteStorageAdapter$$ExternalSyntheticLambda7.run(Unknown Source:12)
W/amplify:aws-datastore(17630):     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:463)
W/amplify:aws-datastore(17630):     at java.util.concurrent.FutureTask.run(FutureTask.java:264)
W/amplify:aws-datastore(17630):     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1137)
W/amplify:aws-datastore(17630):     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:637)
W/amplify:aws-datastore(17630):     at java.lang.Thread.run(Thread.java:1012)
W/amplify:aws-datastore(17630): Caused by: android.database.sqlite.SQLiteConstraintException: UNIQUE constraint failed: TestTableOne.profile_id (code 1555 SQLITE_CONSTRAINT_PRIMARYKEY)
W/amplify:aws-datastore(17630):     at android.database.sqlite.SQLiteConnection.nativeExecute(Native Method)
W/amplify:aws-datastore(17630):     at android.database.sqlite.SQLiteConnection.execute(SQLiteConnection.java:730)
W/amplify:aws-datastore(17630):     at android.database.sqlite.SQLiteSession.execute(SQLiteSession.java:621)
W/amplify:aws-datastore(17630):     at android.database.sqlite.SQLiteStatement.execute(SQLiteStatement.java:47)
W/amplify:aws-datastore(17630):     at com.amplifyframework.datastore.storage.sqlite.SQLCommandProcessor.execute(SQLCommandProcessor.java:78)
W/amplify:aws-datastore(17630):     ... 8 more
NikaHsn commented 6 months ago

Thank you for your patience. The issue has been fixed in version 1.8.0. and I'm closing it. However, if you encounter any issues after updating to version 1.8.0, please don't hesitate to reopen it.