search

objectbox / objectbox-dart

Flutter database for super-fast Dart object persistence
https://docs.objectbox.io/getting-started
Apache License 2.0
927 stars 115 forks source link

Could not begin read transaction (another read transaction is still active on this thread) #288

Closed darshansatra1 closed 2 years ago

darshansatra1 commented 2 years ago

So I'm running a foreground service using the plugin flutter_foreground_task. Before I was using Hive DB, but since the foreground service creates a new isolate I was not able to access the same box in the main isolate, and other isolates since Hive does not support multi-threading so I tried Object Box. It was working awesome and the main isolate was able to see the changes made in other isolates. But the issue began when I found out that Read operations throw an error when another block accesses(Write/Read) the box at the same time. I read the documentation and it says there can be more than one reader, however it was not the case while I was running the foreground service.

Basic info:

[√] Android toolchain - develop for Android devices (Android SDK version 30.0.2) • Android SDK at C:\Users\Darshan\AppData\Local\Android\sdk • Platform android-30, build-tools 30.0.2 • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01) • All Android licenses accepted.

[√] Chrome - develop for the web • Chrome at C:\Program Files\Google\Chrome\Application\chrome.exe

[√] Android Studio (version 4.1.0) • Android Studio at C:\Program Files\Android\Android Studio • Flutter plugin can be installed from: https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)

[√] IntelliJ IDEA Ultimate Edition (version 2021.1) • IntelliJ at C:\Program Files\JetBrains\IntelliJ IDEA 2020.2.3 • Flutter plugin version 57.0.5 • Dart plugin version 211.7665

[√] VS Code (version 1.58.2) • VS Code at C:\Users\Darshan\AppData\Local\Programs\Microsoft VS Code • Flutter extension version 3.24.0

[√] Connected device (3 available) • AC2001 (mobile) • 4f0ec2e7 • android-arm64 • Android 11 (API 30) • Chrome (web) • chrome • web-javascript • Google Chrome 91.0.4472.114 • Edge (web) • edge • web-javascript • Microsoft Edge 92.0.902.55

• No issues found!

### Expected behavior

It was not suppose to throw error since any number of readers are allowed.

### Code

@override List fetchAllHealthData() { try { if (healthDataBox == null) return []; return healthDataBox?.getAll() ?? []; } catch (e) { print(e); return []; } }


- pubspec.yaml.

name: health description:

publish_to: "none"

version: 1.0.5+6

environment: sdk: ">=2.12.0 <3.0.0"

dependencies: flutter: sdk: flutter http: any background_fetch: ^1.0.0 flutter_foreground_task: ^2.0.4 objectbox: ^1.1.1 objectbox_flutter_libs: ^1.1.1

dev_dependencies: flutter_test: sdk: flutter json_serializable: ^4.1.1 build_runner: ^2.0.2 flutter_native_splash: ^1.1.8+4 objectbox_generator: ^1.1.1

flutter_native_splash: color: "#F8F8FC" image: assets/images/splash screen icon.png android: true ios: true

flutter: uses-material-design: true assets:

- Affected entity classes. Although this error happens randomly for all entity

import 'package:objectbox/objectbox.dart';

@Entity() class HealthDataModel { @Id() int id; @Property(type: PropertyType.date) DateTime? date; int? baseStep; int? steps; HealthDataModel({ this.id = 0, this.date, this.baseStep, this.steps, }); }


### Logs, stack traces

I/flutter (31054): ObjectBoxException: failed to create transaction: 10199 Could not begin read transaction (another read transaction is still active on this thread) E/Box (31054): Storage error (code -30783)



### Additional context

Add any other context about the problem here.

- Is there anything special about your app?

It tracks your steps even when the app is killed.

- May transactions or multi-threading play a role?

Yes, isolates are being used that's why multi-threading is required.

- Did you find any workarounds to prevent the issue?

- No
vaind commented 2 years ago

That's pretty weird... Box.getAll() is synchronous and that isn't interrupted so I don't see how anything could try to run on the same thread at the same time. Even though multiple isolates may be scheduled on the same thread by Dart VM, the isolate must "yield" first (using await) before another one can execute.

Can you provide some code actually running the parallel operations? It would be perfect if you could reduce the issue to the minimal flutter app where the issue can be reproduced since I'm not familiar with flutter_foreground_task.

I've had a brief look at flutter_foreground_task and maybe that's even the issue with how the callback is executed directly, not jumping to the UI thread when executing dart's callback? But that's really a stretch since I don't know enough about the plugin and its inner workings... nor about calling dart callbacks from native plugin code (Kotlin) for that matter.

darshansatra1 commented 2 years ago

Okay so what is happening is as the application starts, I start the foreground service. Now as I'm using a pedometer, it emits a stream of steps which we have to listen to. I'm listening to the stream in the main isolate and also in the foreground service. So when a new value occurs in the stream, my foreground service first fetches the previous steps; does some calculation, and then store that calculated steps in the DB and as the main isolate is also listening to the stream, when an event occurs there, I try to fetch the new steps from the DB so that I can show it in the UI. Now as these two the two get events are happening at the same time in different threads, it is throwing me this exception. I would really really appreciate your help here.

vaind commented 2 years ago

Now as these two the two get events are happening at the same time in different threads, it is throwing me this exception.

That's the thing - if they were happening in different threads, then there would be no issue - the error is that it tries to start another TX on the same thread... "(another read transaction is still active on this thread)

OK, I have a clearer picture of what you're doing and with a little more info I may be able to try and reproduce it... Could you please add:

darshansatra1 commented 2 years ago

Foreground Service:

Future<void> startForegroundService({
  required int steps,
  required int yesterdaySteps,
}) async {
  if (await FlutterForegroundTask.isRunningTask) return;
  await FlutterForegroundTask.init(
    notificationOptions: NotificationOptions(
      channelId: 'steps',
      channelName: 'Steps',
      channelImportance: NotificationChannelImportance.NONE,
      priority: NotificationPriority.LOW,
      visibility: NotificationVisibility.VISIBILITY_SECRET,
      iconData: NotificationIconData(
        name: "stat_name",
        resType: ResourceType.mipmap,
        resPrefix: ResourcePrefix.ic,
      ),
    ),
    foregroundTaskOptions: ForegroundTaskOptions(
      autoRunOnBoot: true,
    ),
  );
  await startPeriodicTask(steps: steps, yesterdaySteps: yesterdaySteps);
}

Future<void> startPeriodicTask({
  required int steps,
  required int yesterdaySteps,
}) async {
  await FlutterForegroundTask.start(
    notificationTitle: "Today: $steps steps",
    notificationText: 'Yesterday: $yesterdaySteps steps',
    callback: periodicTaskFun,
  );
}

void periodicTaskFun() async {
  Stream<StepCount> _stepCountStream = Pedometer.stepCountStream;
  StreamSubscription<StepCount>? streamSubscription;
  LocalDataSource? localDataSource;
  FlutterForegroundTask.initDispatcher((timeStamp) async {
    if (streamSubscription != null) {
      return;
    }
    localDataSource = LocalDataSourceImpl();
    await localDataSource!.initialize();
    streamSubscription = _stepCountStream.listen(
      (steps) async { 
        try {
          DateTime date = DateTime(
              steps.timeStamp.year, steps.timeStamp.month, steps.timeStamp.day);
          HealthDataModel healthDataModel;
          List<HealthDataModel> allHealthData =
              localDataSource!.fetchAllHealthData();
          healthDataModel = allHealthData.firstWhere(
              (element) => element.date!.isSameDate(date), orElse: () {
            print("NOT FOUND IN FOREGROUND SERVICE"); 
            HealthDataModel newHealthData;
            newHealthData = HealthDataModel(
              date: date,
              steps: 0, 
            );
            return newHealthData;
          });
          int newSteps = 0;
          if (healthDataModel.baseStep == null) {
            healthDataModel.baseStep = steps.steps;
          }
          if (steps.steps < healthDataModel.baseStep!) {
            int previousSteps = healthDataModel.steps!;
            newSteps = steps.steps + previousSteps;
            healthDataModel.steps = newSteps;
            \\ This is the function getting called to store steps
            await updateHealthData(
                localDataSource: localDataSource!,
                healthDataModel: healthDataModel);
          } else {
            newSteps = steps.steps - healthDataModel.baseStep!;
            if (newSteps != healthDataModel.steps || newSteps == 0) { 
              healthDataModel.steps = newSteps;
              \\ This is the function getting called to store steps
              await updateHealthData(
                localDataSource: localDataSource!,
                healthDataModel: healthDataModel,
              );
            }
          }
          HealthDataModel yesterdayHealthDataModel = allHealthData.firstWhere(
              (element) =>
                  element.date!.compareTo(
                    DateTime.now().subtract(Duration(days: 1)),
                  ) ==
                  0, orElse: () {
            List<String> hoursSteps = [];
            for (int i = 0; i < 24; i++) {
              hoursSteps.add(0.toString());
            }
            return HealthDataModel(
              date: date.subtract(Duration(days: 1)),
              steps: 0,
              hourSteps: hoursSteps,
              activeTime: 0,
              calories: 0,
              distance: 0,
            );
          });
          await FlutterForegroundTask.update(
            notificationTitle: "Today: ${healthDataModel.steps} steps",
            notificationText:
                'Yesterday: ${yesterdayHealthDataModel.steps} steps',
          );
        } catch (e) {
          print(e);
        }
        busy = false;
      },
      cancelOnError: true,
    );
  }, onDestroy: (timeStamp) async {
    localDataSource?.dispose();
    await streamSubscription?.cancel();
  });
}

Future<void> updateHealthData({
  required LocalDataSource localDataSource,
  required HealthDataModel healthDataModel,
}) async {
  await localDataSource.updateHealthDataModel(healthDataModel: healthDataModel);
}

LocalDataSource : 

@override
  Future<HealthDataModel> updateHealthDataModel(
      {required HealthDataModel healthDataModel}) async {
    try {
      if (healthDataBox == null) {
        throw CacheException();
      }
      HealthDataModel model = healthDataBox!.getAll().firstWhere(
          (element) => element.date!.isSameDate(healthDataModel.date!),
          orElse: () {
        return healthDataModel;
      });
      healthDataModel.id = model.id;
      int id = await healthDataBox!.putAsync(healthDataModel);
      healthDataModel.id = id;
      return healthDataModel;
    } catch (e) {
      return healthDataModel;
    }
  }

Main Isolate: So from Main Isolate I'm calling startForegroundService(steps:0,yesterdaySteps:0) to start foreground service. 

 void onStepCount() async {
    // return;
    streamSubscription = _stepCountStream.listen((steps) {
      updateStepsEvent(steps: steps);
    }, cancelOnError: true);
  }

void updateStepsEvent({StepCount steps}){
    \\ This function is calling the function fetchHealthDataFromDate
}

@override
  HealthDataModel? fetchHealthDataFromDate({required DateTime date}) {
    try {
      if (healthDataBox == null) return null;
      int index = healthDataBox!
          .getAll()
          .indexWhere((element) => element.date!.isSameDate(date));
      if (index == -1) return null;
      return healthDataBox!.getAll()[index];
    } catch (e) {
      print(e);
    }
  }

Don't worry about the HealthDataModel, I don't think this issue is because of the model.

darshansatra1 commented 2 years ago

If it helps, I saw the flutter_foreground_task and they're using PluginUilities like this:

options['callbackHandle'] =
          PluginUtilities.getCallbackHandle(callback)?.toRawHandle();
vaind commented 2 years ago

I've tried creating a reproduction in this branch: https://github.com/objectbox/objectbox-dart/compare/288-issue-repro but couldn't get the foreground callback to execute - it just logs FlutterForegroundTask started. but then it doesn't actually call the callback. Can you try to have a look to see what the issue might be, as you're familiar with the plugin?

darshansatra1 commented 2 years ago

Are you trying this in IOS or Android? It won't work in IOS.

I think you're missing some configurations. 

Android 
Since this plugin is based on a foreground service, we need to add the following permission to the AndroidManifest.xml file. Open the AndroidManifest.xml file and specify it between the <manifest> and <application> tags.

<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
And we need to add this permission to automatically resume foreground task at boot time.

<uses-permission android:name="android.permission.RECEIVE_BOOT_COMPLETED" />
And specify the service inside the <application> tag as follows.

<service android:name="com.pravera.flutter_foreground_task.service.ForegroundService" />
vaind commented 2 years ago

Thanks, it was the missing permission setting. Some preliminary info: the callback from the foreground task executes in a separate isolate, thus needs to attach to an existing open store, not open a new one. How do you handle this situation? See this for details: https://stackoverflow.com/a/68519353/2386130

darshansatra1 commented 2 years ago

Okay so the problem is, I'm opening the box twice instead of opening the second box using the reference of the first box?

vaind commented 2 years ago

You can get the port to the background task (and then send over the store reference) like in this commit: https://github.com/objectbox/objectbox-dart/commit/e61ea42c8c0a2826342339b976c129f1b3390c5f

Let me know if that solved your issue.

darshansatra1 commented 2 years ago

Thank you so much, I was gonna give up and move to sqflite, fortunately, this port communication worked and I'm really thankful. Great work and great database.