search

atsign-foundation / at_server

The software implementation of Atsign's core technology
https://docs.atsign.com
BSD 3-Clause "New" or "Revised" License
39 stars 13 forks source link

uptake hive 4.x beta version #2005

Open murali-shris opened 1 week ago

murali-shris commented 1 week ago

Is your feature request related to a problem? Please describe.

hive older version has few issues related to box corruption, updating hive box in isolate.

Describe the solution you'd like

uptake changes in persistence for hive major version and check for performance improvements.

Describe alternatives you've considered

No response

Additional context

No response

murali-shris commented 1 day ago

Progress made on hive 4.x uptake so far 1) Ran into an issue while running dart sample code to open a box IsarNotReadyError: Could not initialize IsarCore library for processor architecture "linux_x64". If you create a Flutter app, make sure to add isar_flutter_libs to your dependencies. For Dart-only apps or unit tests, make sure to place the correct Isar binary in the correct directory.

Invalid argument(s): Failed to load dynamic library 'libisar.so': libisar.so: cannot open shared object file: No such file or directory
#0      initializePlatformBindings (package:isar/src/native/native.dart:26:5)
#1      IsarCore._initialize (package:isar/src/isar_core.dart:42:20)
#2      new _IsarImpl.open (package:isar/src/impl/isar_impl.dart:50:14)
#3      Isar.open (package:isar/src/isar.dart:88:28)
#4      Hive.box (package:hive/src/hive.dart:80:23)
#5      main (file:///home/murali/git/at_protocol/at_server/packages/at_persistence_secondary_server/test/hive_new_test.dart:6:18)
#6      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:297:19)
#7      _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)

Solution: Download isar library from https://github.com/isar/isar/releases/tag/4.0.0-dev.14 and add this line to sample/unit test

Isar.initialize(<path to isar lib>);
e.g in macos
Isar.initialize('libisar_macos.dylib')
  1. generateSecureKey is no longer part of Hive. We use this method to generate a key and encrypt the box. New Hive allows user to pass encryptionKey in String type. There is no explanation in dart doc what type of encryption key can be passed. whether it allows any random string to be used as encryption key ? I have to test this.
  2. Type adapters are no longer part of new Hive. So it is now easier to store and retrieve custom class object. The custom class should have toJson and fromJson impl.
  3. Old Hive.add(..) returns the auto increment key which use as commit log id. New Hive no longer returns auto increment key and doesn't expose any method to retrieve the auto increment key after adding. Commit log logic has to be modified. We have to maintain our own key to maintain latest commit id.
  4. Analysed whether our existing hive box files can be opened with new Hive methods. Not possible since Hive uses .isar extension for new boxes. Hence we have to come up with custom logic to migrate old box data to new box. Will try this migration for these scenarios before I proceed with other code changes for commit log.
    • data migration with simple key-value pair without encryption/type adapter
    • data migration with simple key-value pair plus encryption
    • data migration with custom class and encryption? @gkc @cpswan @cconstab