juliansteenbakker / mobile_scanner

A universal scanner for Flutter based on MLKit. Uses CameraX on Android and AVFoundation on iOS.
BSD 3-Clause "New" or "Revised" License
892 stars 515 forks source link

[Bug] iOS background queue has the wrong QoS priority #1103

Open chenjiangmin opened 4 months ago

chenjiangmin commented 4 months ago

I attempted to introduce this plugin by merely including it in the pubspec.yaml.

I did not use any of its functionalities.

However, the “flutter_baidu_mapapi_map” that I use in my program can no longer work. When opening the map, the thread gets stuck.

Thread running at QOS_CLASS_USER_INTERACTIVE waiting on a lower QoS thread running at QOS_CLASS_DEFAULT.

navaronbracke commented 4 months ago

Could you provide more information? Are you running on MacOS or iOS? What version of mobile_scanner are you using?

chenjiangmin commented 4 months ago

Yes, I am using the iOS system to debug the system version, which is iOS 17

[✓] Flutter (Channel stable, 3.19.6, on macOS 14.5 23F79 darwin-arm64, locale zh-Hans-CN)
    • Flutter version 3.19.6 on channel stable at /Users/chenjiangmin/fvm/versions/3.19.6
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 54e66469a9 (10 周前), 2024-04-17 13:08:03 -0700
    • Engine revision c4cd48e186
    • Dart version 3.3.4
    • DevTools version 2.31.1

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
    • Android SDK at /Users/chenjiangmin/Library/Android/sdk
    • Platform android-34, build-tools 33.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.9+0-17.0.9b1087.7-11185874)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.4)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15F31d
    • CocoaPods version 1.15.2

[✓] Android Studio (version 2023.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.9+0-17.0.9b1087.7-11185874)

[✓] IntelliJ IDEA Ultimate Edition (version 2023.2.5)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] VS Code (version 1.90.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.90.0
chenjiangmin commented 4 months ago

WeChat816100404820b5b153945c3a79d07ff2

Introducing this plugin will cause iOS to freeze when running to the bottom of the map, and the xcode will stop at this position Is there any conflict in this inventory

navaronbracke commented 4 months ago

@chenjiangmin I think this has to do with the priorities of the threads in the plugin. We use both DispatchQueue.main for things that need to happen on the main thread (i.e. interacting with the camera)

We also use the main thread for notifying about video frames here: https://github.com/juliansteenbakker/mobile_scanner/blob/master/ios/Classes/MobileScanner.swift#L238

However, we also have a different thread, to allow processing to happen on a background thread: https://github.com/juliansteenbakker/mobile_scanner/blob/master/ios/Classes/MobileScanner.swift#L54

Do you have information, indicating that the thread with label camera-handling is causing this hang?

According to the docs, the constructor uses .unspecified as QoS argument: https://developer.apple.com/documentation/dispatch/dispatchqueue/2300059-init

I don't really know what QoS type we should use here, if you could verify which of the following values would fix the priority issue?

https://developer.apple.com/documentation/dispatch/dispatchqos#3126094

navaronbracke commented 4 months ago

On MacOS we use the global queue with .userInitiated https://github.com/juliansteenbakker/mobile_scanner/blob/master/macos/Classes/MobileScannerPlugin.swift#L120C13-L120C54

Perhaps iOS should do that as well?

navaronbracke commented 4 months ago

@ened Do you remember why the QoS was omitted in https://github.com/juliansteenbakker/mobile_scanner/commit/cd506ebde87ad3219071cb2308b2ba44a64fd205 ? Or was this just an oversight?

navaronbracke commented 4 months ago

@chenjiangmin Could you test out this patch? If this does not work, then we should evaluate if we should bump it to .userInteractive, although I think that priority will not be the right fit here.

dependencies:
  mobile_scanner:
    git:
      url: https://github.com/navaronbracke/mobile_scanner.git
      ref: fix_ios_queue_qos_value
chenjiangmin commented 4 months ago

Sorry, I just tried pulling this branch now, but it seems to have no effect and still stuck the thread

navaronbracke commented 4 months ago

@chenjiangmin Can you pull the branch again? I made another change, switching from .userInitiated to .userInteractive.

chenjiangmin commented 4 months ago

image

After I pulled it again, I saw that the code was still the same as before

navaronbracke commented 4 months ago

Per this commit https://github.com/juliansteenbakker/mobile_scanner/pull/1110/commits/9959c42188ad0c03f27f7b6870c28131e85bf2ed

you should be seeing .userInteractive ? Perhaps doing a flutter clean && flutter pub get fixes that?

chenjiangmin commented 4 months ago

image

I updated the plugin code, but when I open the map, it still has the same effect. It's normal not to introduce this plugin for the map. I'm not using the functionality of this plugin now

image
navaronbracke commented 4 months ago

Since with this patch, mobile_scanner is not using .default as QoS value, perhaps it is a bug with that map plugin? They should not be using .default either for their dispatch queues. Unfortunately I can't see their source code so I can't tell which of their threads is causing the issue.

navaronbracke commented 3 months ago

This seems to be a bug with Flutter's raster thread, which was only surfaced in newer XCode versions, and might be related to using platform views.

The underlying bug was fixed on Flutter's master channel.

@chenjiangmin Can you retest on the Flutter master channel, to see if the issue still occurs?

I linked the underlying issue in the comment when my PR was closed.