search

krille-chan / fluffychat

The cutest instant messenger in the [matrix]
https://fluffychat.im/
GNU Affero General Public License v3.0
1.2k stars 217 forks source link

💽 New Matrix Dart SDK Database Feedback #735

Closed krille-chan closed 10 months ago

krille-chan commented 10 months ago

With the newest release candidate v1.16.1 there is a new database in place which will replace the current HiveCollections database. The new database is using Sqlite3 on native and IndexedDB on web. The app should automatically migrate the data without the need to re-login. The app will just need a few seconds more to open and display a notification in the meantime. It might happen that you get a prompt to reconnect your chat backup but that should disappear after closing and opening the app.

The open questions are:

  1. Is the database migration working for everyone? Is the migration notification shown on all platforms?
  2. How is the new database performing? Is it faster or slower than before?
  3. Are there bugs with the new database?

Any feedback would be appreciated! Please also share as much information about your device as possible (Operating system? OS version? Device type?)

How to join the public beta

iOS and iPadOS

You can install the latest release candidate of FluffyChat by using TestFlight:

https://testflight.apple.com/join/fhMtOXc0

Please note: Apple reviews all apps in public TestFlight manually. This can take several days so the new version might not be out yet at the time you see this post.

Android

PlayStore

You can join the public beta program in the PlayStore:

https://play.google.com/apps/testing/chat.fluffy.fluffychat

Please note: Google reviews all apps in the PlayStore Beta track manually. This can take several days so the new version might not be out yet at the time you see this post.

Linux Desktop (including Linux Phones)

You can just use the candidate channel of our Snap version:

https://snapcraft.io/install/fluffychat/ubuntu

Also the edge channel contains the latest builds from main if you like.

Web

Just use https://fluffychat.im/nightly

joeldebruijn commented 10 months ago

Signed up for Android beta👍

LFd3v commented 10 months ago

I can test on Linux and Android, any chance of providing download links for the apps (I do not use Flatpaks or Snaps, on both Android and Linux I simply install the binaries from the Releases page).

Edit: about the Web app, one must first login using the stock URL (fluffychat.im/web) do some tests and then open fluffychat.im/nightly and it will migrate the db as well or the migration is not done on web?

Thanks!

redsky17 commented 10 months ago

Hey @krille-chan, just following up from our discussion on the fediverse... So I'm running on iPhone 12 Pro Max, OS version 17.1.2. The first time I launched fluffychat following the testflight enrollment, the UI locked up and eventually fluffychat crashed with no notice. Subsequent launches 'kind of' work in that my accounts aren't obviously broken or anything. The room list itself loads, but any sort of interaction is more or less immediately met with a frozen UI. I have two accounts on this instance of fluffychat: @red_sky:nheko.im and @red_sky:ocean.joedonofry.com. The former account is the larger of the two, with somewhere around 100 rooms or so. The second is smaller, with only maybe two dozen or so rooms. The second account is also the current 'active' account when I'm using the UI and it freezes. I've also been using fluffychat with the second account for probably two or more years, whereas the first account is only logged in as of maybe two weeks ago.

You also asked for a screenshot, but I'm not sure exactly what you're looking for. Can you elaborate? Let me know what else you need! Thanks

krille-chan commented 10 months ago

I've created a release here: https://github.com/krille-chan/fluffychat/releases/tag/rc1.16.1-1 in a few minutes there should be an APK and Linux binary available there. For web, just visit https://fluffychat.im/nightly you can try the migration by logging in at https://fluffychat.im/web and then open the nightly app. Please notice that the migration does not work vice versa so once you have migrated your session, you are just not logged in at https://fluffychat.im/web

Sir-Photch commented 10 months ago

Works with me on the github release. Some minor nltification localization issue though: "Die Datenbank ist optimiert" should be "Die Datenbank wird optimiert"

TheArcaneBrony commented 10 months ago

Was this by chance released with 1.16.0? We got logged out of all sessions, and the app has been feeling snappier, though I'd attributed that to not logging into my synapse account, but rather using conduit.

joeldebruijn commented 10 months ago

Signed up for Android beta👍

1.16.1 just came in, took a few extra seconds (just 5 rooms and 2 private chats). Had to tap them once to reload messages. Worked fine (at least opening and getting new messages).

krille-chan commented 10 months ago

1.16.0 shouldnt have got it. But have heard it multiple times now that aessuons got lost there. Maybe a hive problem. Really frustrating...

I found one bug in the new database regarding deleting messages:

ERROR] Error during processing events - Unsupported operation: Cannot remove from an unmodifiable list
#0      UnmodifiableListMixin.removeWhere (dart:_internal/list.dart:139)
#1      MatrixSdkDatabase.removeEvent (package:matrix/src/database/matrix_sdk_database.dart:813)
<asynchronous suspension>
#2      MatrixSdkDatabase.storeEventUpdate (package:matrix/src/database/matrix_sdk_database.dart:1067)
<asynchronous suspension>
#3      Client._handleRoomEvents (package:matrix/src/client.dart:2098)
<asynchronous suspension>
#4      Client._handleRooms (package:matrix/src/client.dart:1965)
<asynchronous suspension>
#5      Client._handleSync (package:matrix/src/client.dart:1811)
<asynchronous suspension>
#6      Client._innerSync.<anonymous closure> (package:matrix/src/client.dart:1742)
<asynchronous suspension>
#7      BoxCollection.transaction.<anonymous closure> (package:matrix/src/database/sqflite_box.dart:90)
<asynchronous suspension>
#8      BoxCollection.transaction (package:matrix/src/database/sqflite_box.dart:83)
<asynchronous suspension>
#9      Client._innerSync.<anonymous closure> (package:matrix/src/client.dart:1749)
<asynchronous suspension>
#10     Client._innerSync (package:matrix/src/client.dart:1747)
<asynchronous suspension>
#11     Client._sync.<anonymous closure> (package:matrix/src/client.dart:1663)
<asynchronous suspension>

Should be easy to fix 😊💪

Rhaletskyj commented 10 months ago

thx

krille-chan commented 10 months ago

Found another oopsie when using the web version:

dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 127:3        castError
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/operations.dart 818:12   cast
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/classes.dart 652:14      as_C
packages/matrix/src/database/matrix_sdk_database.dart 506:35                       <fn>
dart-sdk/lib/internal/iterable.dart 403:20                                         moveNext
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/operations.dart 1277:20  next
dart-sdk/lib/_internal/js_dev_runtime/patch/core_patch.dart 555:14                 of
LFd3v commented 10 months ago

I've created a release here: https://github.com/krille-chan/fluffychat/releases/tag/rc1.16.1-1 in a few minutes there should be an APK and Linux binary available there. For web, just visit https://fluffychat.im/nightly you can try the migration by logging in at https://fluffychat.im/web and then open the nightly app. Please notice that the migration does not work vice versa so once you have migrated your session, you are just not logged in at https://fluffychat.im/web

Thank you for the binaries.

Linux

  1. I have used a test account, upgraded from v1.16.0 to rc1.16.1-1. After starting the rc1 version from the terminal, it showed:
Click to expand ``` $ ./fluffychat (fluffychat:62841): Gdk-CRITICAL **: 12:38:49.061: gdk_window_get_state: assertion 'GDK_IS_WINDOW (window)' failed flutter: [Matrix] Welcome to FluffyChat <3 [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: Invalid argument(s): Linux settings must be set when targeting Linux platform. #0 FlutterLocalNotificationsPlugin.initialize (package:flutter_local_notifications/src/flutter_local_notifications_plugin.dart:177) #1 ClientManager.sendMigrationNotification (package:fluffychat/utils/client_manager.dart:141) #2 ClientManager.getClients.. (package:fluffychat/utils/client_manager.dart:52) #3 Client._migrateFromLegacyDatabase (package:matrix/src/client.dart:3064) #4 Client.init (package:matrix/src/client.dart:1523) #5 Future.wait. (dart:async/future.dart:523) #6 ClientManager.getClients (package:fluffychat/utils/client_manager.dart:45) #7 main (package:fluffychat/main.dart:26) flutter: dynamic_color: Accent color detected. ```

and the app stayed in the refresh state, so I closed it after about 5min and it started fine afterwards.

  1. Typing in the search or message box spams this error message (I think it did the same in v1.16.0 tho), without (visible) side effects:
Click to expand ``` [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: Null check operator used on a null value #0 _isPressed. (package:keyboard_shortcuts/keyboard_shortcuts.dart:51) #1 WhereIterator.moveNext (dart:_internal/iterable.dart:451) #2 _isPressed (package:keyboard_shortcuts/keyboard_shortcuts.dart:55) #3 _KeyBoardShortcuts.listener (package:keyboard_shortcuts/keyboard_shortcuts.dart:135) #4 RawKeyboard.handleRawKeyEvent (package:flutter/src/services/raw_keyboard.dart:705) #5 KeyEventManager.handleRawKeyMessage (package:flutter/src/services/hardware_keyboard.dart:1050) #6 BasicMessageChannel.setMessageHandler. (package:flutter/src/services/platform_channel.dart:223) #7 _DefaultBinaryMessenger.setMessageHandler. (package:flutter/src/services/binding.dart:567) #8 _invoke2 (dart:ui/hooks.dart:344) #9 _ChannelCallbackRecord.invoke (dart:ui/channel_buffers.dart:45) #10 _Channel.push (dart:ui/channel_buffers.dart:135) #11 ChannelBuffers.push (dart:ui/channel_buffers.dart:343) #12 PlatformDispatcher._dispatchPlatformMessage (dart:ui/platform_dispatcher.dart:722) #13 _dispatchPlatformMessage (dart:ui/hooks.dart:257) ```
  1. Trying to start a call crashes the app:
Warning: Failed to respond to a message. This is a memory leak.
[1]    63827 segmentation fault (core dumped)  ./fluffychat

  1. New stories created on v1.16.0 do not seem to work on rc1: the user is invited to a new room but messages cannot be decrypted. Stories migrated from the previous version display as status on rc1, even if the user is not online (not sure if this is intended).

  2. Pinning/unpinning messages work, but clicking on the pinned message scrolls to the wrong position (not sure if this is a regression or already present in v1.16.0).

Deleting, quoting, replying messages, as well as blocking/unblocking users seem to work fine. Please feel free to ask for any specific tests if needed. I will test rc1 on Android soon.

PS: I did not notice this in the past, so I guess this is new: the notification now shows an option to open the message and another to mark it read. This is great! I hope this also works on mobile.

Cheers

LFd3v commented 10 months ago

Android

  1. Same test account, upgraded from v1.16.0 to rc1.16.1-1, it showed a notification about the database migration and then, after a short while, it displayed all rooms normally, no need to restart the app.
  2. In app log did not show any errors, so I am not sure if the error when typing text happens or not.
  3. Starting a call does not crash the app, but it seems that causes some problem afterwards since I cannot change a status message from my account and opening rooms with unread messages does not clear the unread status... Once the app displayed "Something went wrong...", and only restarting the app and clearing the cache fixed the problem.
  4. The same happens related to stories/status as the desktop version.
  5. Same for pins, but after a while I found out that if all options to hide stuff in the room are disabled then it scrolls to the actual message.

Sadly it seems that notifications on Android don't have the open message/mark read as the desktop version. Other than this, I think there is some problem with the cache in the new version, besides the problem with calls, because after I force close the app and restart it it will not show some profile pictures after the app is launched, but opening any room and then going back display them properly.

krille-chan commented 10 months ago

Thanks for the tests <3 about some things I can at least say that they have nothing to do with the database:

So looks like I have to do some fixes for the Linux version then next

LFd3v commented 10 months ago

@krille-chan Thank you for the feedback.

I kept the rc1 version on Android but I had to downgrade to v1.16.0 yesterday since I could not get the app working again:

  1. Open Settings > Security
  2. Tap on App Lock
  3. Force close the app (I tapped on App Lock by mistake, I already had a PIN and did not want to change it)
  4. After restarting the app, it showed the error "Oops, something went wrong" (or something like this)
  5. I opened a chat and used /clearcache
  6. Restarted the app but it got even worse since I could not open any rooms, all visible info was gone

The good news is that it seems that the migration did not delete the old data (intended?) since after downgrading to v1.16.0 I did not have to login or verify the app again. I tried reproducing the same steps above and no problems on v1.16.0. Maybe it was a coincidence that I had the App Lock dialog open, and simply force closing the app corrupted the database, but clearing the cache did not help, it actually made things worse. The fact is that, as I said previously, I already had noticed some problem with the data cache on Android using the rc version.

Anyway, I hope this report helps somehow. Once a new rc is released, I can do the same tests again.

Cheers

krille-chan commented 10 months ago

I found another edge case where messages can get lost on limited timeline flag sent by the server. Has been fixed in matrix dart sdk v0.25.3. I will push a new rc for FluffyChat v1.17.0 now

TheOneWithTheBraid commented 10 months ago

Issue on Linux database migration :

Flutter logs ``` (fluffychat:12505): Gdk-CRITICAL **: 17:31:02.743: gdk_window_get_state: assertion 'GDK_IS_WINDOW (window)' failed flutter: [Matrix] Welcome to FluffyChat <3 flutter: [Matrix] Initialize client FluffyChat linuxDebug flutter: [Matrix] Benchmark: Build database -> 91 ms flutter: [Matrix] Benchmark: Get Client from store -> 4 ms flutter: [Matrix] Successfully connected as alice with http://localhost flutter: [Matrix] Store session in backup flutter: [Matrix] FluffyChat started in foreground mode. Rendering GUI... Syncing files to device Linux... flutter: [Matrix] Benchmark: Get all account data from store -> 77 ms flutter: [Matrix] The user has been logged out! flutter: [Matrix] Benchmark: Get all user device keys from store -> 128 ms flutter: [Matrix] Benchmark: Get room list from store -> 128 ms Syncing files to device Linux... 181ms Flutter run key commands. r Hot reload. 🔥🔥🔥 R Hot restart. h List all available interactive commands. d Detach (terminate "flutter run" but leave application running). c Clear the screen q Quit (terminate the application on the device). A Dart VM Service on Linux is available at: http://127.0.0.1:35115/DWRnx_nZgxM=/ flutter: [Matrix] Set background mode in VOIP plugin - false flutter: [Matrix] [VOIP] Initialized flutter: dynamic_color: Accent color detected. The Flutter DevTools debugger and profiler on Linux is available at: http://127.0.0.1:9100?uri=http://127.0.0.1:35115/DWRnx_nZgxM=/ flutter: [Matrix] AppLifecycleState = AppLifecycleState.inactive flutter: [Matrix] Set background mode in VOIP plugin - false ```

I will take a look to get further logs from it. As of now happened on all my Linux devices. I know these logs are not helpful for debugging, it's just meant as information that the change is currently breaking non-snap systems.

krille-chan commented 10 months ago

Got released now 🥳