BarcodeScan on iOS not working with different iPhones/iOS versions

gluonhq / attach

GNU General Public License v3.0

49 stars 26 forks source link

BarcodeScan on iOS not working with different iPhones/iOS versions #322

Closed molbertz closed 2 years ago

molbertz commented 2 years ago

Hello there,

I noticed the barcode scan service is not working reliably anymore. When invoked, the permission alert is displayed but the preview view doesn't open. This happens on a variety of iPhone/iPad/iOS combinations. As far as I can tell the log doesn't show anything that could lead to finding out why the view doesn't open; No visible errors etc.

Here is an overview of what I tested:

Hardware	iOS	Result
iPhone 12 Pro	15.6.1	No camera view
iPhone 13 Pro	15.6.1	No camera view
iPad 6th Gen	15.6.1	Camera view opens
iPad Air 4th Gen	15.6.1	Camera view opens
iPhone 11 Pro	15.6.1/16.0 Beta	Camera view opens

I tested with the most recent attach library version 4.0.15 and compiled using GraalVM 22.1.0.1 Java 17 CE. I also tried GraalVM 22.1.0.1 Java 11. The app I tested with was the HelloGluon App from the gluon-samples. Instead of a System.out when pressing the Search button I invoke the scan service.

Thank you for your time, Michael

jperedadnr commented 2 years ago

Thanks for the report, 4.0.15 already includes the changes we discussed earlier. Since we can't test ourselves, is it possible that you can build Attach with some extra debug information and test directly on your devices? We can help on the build if needed.

jperedadnr commented 2 years ago

Also, since you are at it, can you try the camera with the PicturesService? It does something similar as the Barcode scanner, so it might also fail.

molbertz commented 2 years ago

The fix we discussed earlier is working. I don't think that has anything to do with the new problem. IIRC I got the Attach build working. I'll try to add some more debug info. I'll look into the PictureService as well if I find some time.

molbertz commented 2 years ago

Okay,

So I have compiled the attach lib and tested the app. Same behavior. The native code runs normally. All my new logging shows up the same. Yet, on iPhone 13 the camera view screen does not open. No problems on iPhone X.

What is interesting is the native code itself seems to work. I copied it to a new native app and it works on every phone. Could you maybe have a look at how you are compiling the library?

As for the PicturesService; same story. Works on iPhone X not on iPhone 13. The permission request is the only thing that pops up once. BTW: When you send the apps to the background and bring them back up again the camera preview view appears. But buttons like "cancel" are unresponsive.

I'm interested to learn what you can find out about this.

jperedadnr commented 2 years ago

Thanks for testing!

Your comments are in line to what others have observed too.

It looks somehow that the JavaFX view (GlassWindow) that used to get covered by the camera view, now remains on top, and while the sublayer with the camera is still there and working, it is not visible by the user anymore.

This would be good to get confirmed (i.e logging all the views that are in place. If you have the chance, add after this line:

NSLog(@"tree view is %@", [view recursiveDescription]);

and then after the scanning service has started.

(see also this work in progress to clarify about how to get the GlassWindow view).

Nothing has changed in terms of API for the AVCaptureSession, as far as I know, and given the fact that the camera service also fails (which uses UIImagePickerController), it looks more like an issue in GlassWindow, with possibly some deprecated method that has different behaviour on new devices...

molbertz commented 2 years ago

That sounds plausible that the wrong view remains on top. Especially because if you press the home button to send the app to the background and then open it again, the camera view does appear. But buttons on it like "cancel" aren't responsive.

I tried putting the requested logging calls in but it seems the method is not known:

> Task :barcode-scan:iosBuild
native build for BarcodeScan started
/Users/bscdev/Temp/clone_attach/attach/modules/barcode-scan/src/main/native/ios/BarcodeScan.m:148:44: warning: instance method '-recursiveDescription' not found (return type defaults to 'id') [-Wobjc-method-access]
    AttachLog(@"1 true view is: %@", [view recursiveDescription]);

Is this maybe a custom category you wrote?

I tested it anyway and the loggings look like this:

Aug 29 06:01:41 BSC-Connect-Mobile[10750] <Notice>: [AttachLog] 1 true view is: <UIView: 0x1230089f0; frame = (0 0; 390 844); autoresize = W+H; layer = <CALayer: 0x28246c6a0>>
Aug 29 06:01:41 BSC-Connect-Mobile(UIKitCore)[10750] <Notice>: Evaluating dispatch of UIEvent: 0x281134600; type: 0; subtype: 0; backing type: 11; shouldSend: 1; ignoreInteractionEvents: 0, systemGestureStateChange: 0
Aug 29 06:01:41 BSC-Connect-Mobile(UIKitCore)[10750] <Notice>: Sending UIEvent type: 0; subtype: 0; to windows: 1
Aug 29 06:01:41 BSC-Connect-Mobile(UIKitCore)[10750] <Notice>: Sending UIEvent type: 0; subtype: 0; to window: <GlassMainWindow: 0x123333810>; contextId: 0x2FA90400
Aug 29 06:01:41 BSC-Connect-Mobile(UIKitCore)[10750] <Notice>:     event = TouchEventsCompleted;
Aug 29 06:01:41 BSC-Connect-Mobile(UIKitCore)[10750] <Notice>: }
Aug 29 06:01:41 BSC-Connect-Mobile(UIKitCore)[10750] <Notice>:     event = TouchEventsCompleted;
Aug 29 06:01:41 BSC-Connect-Mobile(UIKitCore)[10750] <Notice>: }
Aug 29 06:01:41 BSC-Connect-Mobile(UIKitCore)[10750] <Notice>: <XCSynthesizedEventRecord 'Single-Finger Touch Action display 0'>

// ...

Aug 29 06:01:41 BSC-Connect-Mobile(AVFCapture)[10750] <Notice>: <<<< AVCaptureFigVideoDevice >>>> -[AVCaptureFigVideoDevice _setActiveVideoMinFrameDurationInternal:]: MinFrameDuration to set 1 / 30
Aug 29 06:01:41 BSC-Connect-Mobile(AVFCapture)[10750] <Notice>: <<<< AVCaptureFigVideoDevice >>>> -[AVCaptureFigVideoDevice _setActiveVideoMaxFrameDurationInternal:]: MaxFrameDuration to set 1 / 30
Aug 29 06:01:41 BSC-Connect-Mobile(PrototypeTools)[10750] <Notice>: Not observing PTDefaults on customer install.
Aug 29 06:01:41 BSC-Connect-Mobile(AVFCapture)[10750] <Notice>: <<<< AVCaptureFigVideoDevice >>>> -[AVCaptureFigVideoDevice _setActiveVideoMinFrameDurationInternal:]: MinFrameDuration to set 1 / 30
Aug 29 06:01:41 BSC-Connect-Mobile(AVFCapture)[10750] <Notice>: <<<< AVCaptureFigVideoDevice >>>> -[AVCaptureFigVideoDevice _setActiveVideoMaxFrameDurationInternal:]: MaxFrameDuration to set 1 / 30
Aug 29 06:01:41 BSC-Connect-Mobile[10750] <Notice>: [AttachLog] 2 true view is: <UIView: 0x1230089f0; frame = (0 0; 390 844); autoresize = W+H; layer = <CALayer: 0x28246c6a0>>

The work in progress way resulted in two different logs:

Aug 29 06:27:25 BSC-Connect-Mobile[10942] : got view: > with _firstView: >, _secondView > and _glassView >

and when I tried the second time I got this:

Aug 29 06:36:41 BSC-Connect-Mobile[11016] : got view: > with _firstView: >, _secondView views="NO;" layer="> and _glassView views="NO;" layer=">

Also I couldn't put the if ([_glassView isKindOfClass:[GlassWindow class]]) { call in because GlassWindow.h is not found. I tried to assign the _glassView to view anyway but is also doesn't work. Camera preview is not showing.

I wish I could be of more help but this is how far I've gotten. Let me know if I can do anything about these problems on my end.

jperedadnr commented 2 years ago

recursiveDescription should work for you. It is not well documented, but you can find it here: https://developer.apple.com/library/archive/technotes/tn2239/_index.html or if you search for debugging on iOS.

About GlassWindow, I added the .h, https://github.com/jperedadnr/attach-2/blob/85a57acff9e691390f51d811c8ea22b433a9b83e/gradle/include/ios/GlassWindow.h but that's not really needed now. If you want to use it, you'll need also this.

However, I see now something different testing on my iPhoneX:

with

    // get the view
    view = self.view;
    NSLog(@"tree view is %@", [view recursiveDescription]);

I get:

<UIView: 0x1074082c0; frame = (0 0; 375 758); autoresize = W+H; layer = <CALayer: 0x283c3dd60>>

but when using this instead:

NSArray *views = [[[UIApplication sharedApplication] keyWindow] subviews];
    view = views[0]; // <-----
    NSLog(@"tree view is %@", [view recursiveDescription]);

I get:

  tree view is <UITransitionView: 0x105307360; frame = (0 0; 375 812); autoresize = W+H; layer = <CALayer: 0x2838bd320>>
    | <UIDropShadowView: 0x1053125d0; frame = (0 0; 375 812); autoresize = W+H; layer = <CALayer: 0x2838bd300>>
    |    | <GlassMainView: 0x10540ab70; frame = (0 0; 375 812); autoresize = W+H; autoresizesSubviews = NO; layer = <CALayer: 0x283886b40>>
    |    |    | <GlassWindow: 0x1053093a0; frame = (0 0; 375 812); autoresizesSubviews = NO; layer = <CALayer: 0x2838be720>>
    |    |    |    | <UIView: 0x105328e20; frame = (0 0; 375 812); autoresizesSubviews = NO; layer = <CALayer: 0x2838be6c0>>
    |    |    |    |    | <GlassViewGL: 0x105876200; baseClass = UIScrollView; frame = (0 0; 375 812); clipsToBounds = YES; opaque = NO; gestureRecognizers = <NSArray: 0x283688240>; layer =

Which shows the expected hierarchy of views, and it still works fine when the camera layer view is added (with a minor side effect for now, the Cancel button is visible but can't be tapped).

So can you try this very simple change?

+    NSArray *views = [[[UIApplication sharedApplication] keyWindow] subviews];
-    view = self.view;
+    view = views[0];

-   [view addSubview:navBar];
+   [self.view addSubview:navBar];

molbertz commented 2 years ago

But according to Apple it is just a method that can be used while debugging in xCode? How do you use it in NSLog? Anyway I tried the change: The local view is now the first view of the NSArray and the navBar is now added to self.view. But it's also not working.

BTW since you mentioned "Deprecated Methods" earlier. That [[UIApplication sharedApplication] keyWindow] seems to be deprecated as well.

jperedadnr commented 2 years ago

It can be used for debugging from command line too. I don't plan to use that in production, but it comes handy for testing.

It can be helpful in this case for instance, if we can figure out the issue with view/layers that are displayed before and after the camera view is opened.

Could you run with that debug info for both iPhone 11 and 13 and see if you notice any difference?

If recursiveDescription doesn't print enough debug info, you can always try to go through the hierarchy UIView/UIWindow/CALayer and try to print out their visibility status.

jperedadnr commented 2 years ago

With some minor changes I have a more "complete" log of views:

[rootViewController presentViewController:self animated:YES completion:^(void) {
        NSLog(@"Completed presenting presentViewController");
        NSLog(@"self.view:\n%@", [self.view recursiveDescription]);
        NSLog(@"view[0]:\n%@", [view recursiveDescription]);
    }];

// init view

 self.view is:
 <UIView: 0x109207790; frame = (0 0; 375 812); autoresize = W+H; layer = <CALayer: 0x280bc0400>>

 view[0] is:
 <UITransitionView: 0x1090255a0; frame = (0 0; 375 812); autoresize = W+H; layer = <CALayer: 0x280be8180>>
    | <UIDropShadowView: 0x109024810; frame = (0 0; 375 812); autoresize = W+H; layer = <CALayer: 0x280bea580>>
    |    | <GlassMainView: 0x109014d20; frame = (0 0; 375 812); autoresize = W+H; autoresizesSubviews = NO; layer = <CALayer: 0x280bece60>>
    |    |    | <GlassWindow: 0x109025760; frame = (0 0; 375 812); autoresizesSubviews = NO; layer = <CALayer: 0x280bd0620>>
    |    |    |    | <UIView: 0x109024a30; frame = (0 0; 375 812); autoresizesSubviews = NO; layer = <CALayer: 0x280bd3480>>
    |    |    |    |    | <GlassViewGL: 0x10883f800; baseClass = UIScrollView; frame = (0 0; 375 812); clipsToBounds = YES; opaque = NO; gestureRecognizers = <NSArray: 0x2805d6730>; layer = <CAEAGLLayer: 0x2805d6100>; contentOffset: {562.66666666666663, 1218}; contentSize: {1500, 3248}; adjustedContentInset: {44, 0, 34, 0}>

  // av running 
  self.view:
 <UIView: 0x109207790; frame = (0 0; 375 758); autoresize = W+H; layer = <CALayer: 0x280bc0400>>
    | <UINavigationBar: 0x1091520b0; frame = (0 44; 375 44); opaque = NO; layer = <CALayer: 0x280bfebc0>> no-scroll-edge-support
    |    | <_UIBarBackground: 0x109152500; frame = (0 0; 375 56); userInteractionEnabled = NO; layer = <CALayer: 0x280bfed80>>
    |    |    | <UIImageView: 0x109155dc0; frame = (0 0; 375 56); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0x280bf4640>>
    |    |    | <_UIBarBackgroundShadowView: 0x109050730; frame = (0 56; 375 0); layer = <CALayer: 0x280bf1460>> clientRequestedContentView effect=none
    |    |    |    | <_UIBarBackgroundShadowContenx109152970
    |    |    | <_UIButtonBarStackView: 0x1091552a0; frame = (8 6; 74.6667 44); layer = <CALayer: 0x280bf4120>> buttonBar=0x2834c8a50
    |    |    |    | <_UIButtonBarButton: 0x109158430; frame = (0 0; 74.6667 44); tintColor = <UIDynamicSystemColor: 0x281ec3600; name = systemBlueColor>; gestureRecognizers = <NSArray: 0x28059c7b0>; layer = <CALayer: 0x280bf5580>>
    |    |    |    |    | <_UIModernBarButton: 0x109158b50; frame = (8 7.33333; 66.6667 26); opaque = NO; userInteractionEnabled = NO; 10914ee70; frame = (130.333 14; 114.333 25.3333); text = 'Badge Scan'; userInteractionEnabled = NO; layer = <_UILabelLayer: 0x2828fe940>>
    |    | <UIView: 0x1090514f0; frame = (0 0; 0 0); userInteractionEnabled = NO; layer = <CALayer: 0x280bd9b60>>

view[0]:
 <UITransitionView: 0x1090255a0; frame = (0 0; 375 812); autoresize = W+H; layer = <CALayer: 0x280be8180>>
    | <UIDropShadowView: 0x109024810; frame = (16 44; 343 742.709); transform = [0.91466666666666663, 0, 0, 0.91466666666666663, 0, 9.3546666666666738]; clipsToBounds = YES; autoresize = W+H; layer = <CALayer: 0x280bea580>>
    |    | <GlassMainView: 0x109014d20; frame = (0 0; 375 812); autoresize = W+H; autoresizesSubviews = NO; layer = <CALayer: 0x280bece60>>
    |    |    | <GlassWindow: 0x109025760; frame = (0 0; 375 812); autoresizesSubviews = NO; layer = <CALayer: 0x280bd0620>>
    |    |    |    | <UIView: 0x109024a30; frame = (0 0; 375 812); autoresizesSubviews = NO; layer = <CALayer: 0x280bd3480>>
    |    |    |    |    | <GlassViewGL: 0x10883f800; baseClass = UIScrollView; frame = (0 0; 375 812); clipsToBounds = YES; opaque = NO; gestureRecognizers = <NSArray: 0x2805d6730>; layer = <CAEAGLLayer: 0x2805d6100>; contentOffset: {562.66666666666663, 1218}; contentSize: {1500, 3248}; adjustedContentInset: {44, 0, 34, 0}>
    |    | <UIDimmingView: 0x10904dad0; frame = (-375 -812; 1125 2436); opaque = NO; gestureRecognizers = <NSArray: 0x2805d7a50>; layer = <CALayer: 0x280bda540>>
    | <AVCaptureVideoPreviewLayer: 0x28059a490> (layer)
    |    | <CALayer: 0x280bfe2a0> (layer)

Notice the UIDimmingView... this post explains it: https://itnext.io/ios-15-uisheetpresentationcontroller-d469d144a520

Would be good to get the same from your side.

(this is my diff: patch.txt)

molbertz commented 2 years ago

I applied you patch and this is what I've got for the iPhone 13 Pro (iOS 15.6):

Aug 30 10:02:00 BSC-Connect-Mobile[4741] <Notice>: 1 self.view:
Aug 30 10:02:00 BSC-Connect-Mobile[4741] <Notice>: <UIView: 0x15425b7e0; frame = (0 0; 390 844); autoresize = W+H; layer = <CALayer: 0x282e46200>>
Aug 30 10:02:00 BSC-Connect-Mobile[4741] <Notice>: 1 tree view:
Aug 30 10:02:00 BSC-Connect-Mobile[4741] <Notice>: <UITransitionView: 0x154331290; frame = (0 0; 390 844); autoresize = W+H; layer = <CALayer: 0x282e11ce0>>
Aug 30 10:02:00 BSC-Connect-Mobile[4741] <Notice>:    | <UIDropShadowView: 0x1543316d0; frame = (0 0; 390 844); autoresize = W+H; layer = <CALayer: 0x282e11d00>>
Aug 30 10:02:00 BSC-Connect-Mobile[4741] <Notice>:    |    | <GlassMainView: 0x15422ccc0; frame = (0 0; 390 844); autoresize = W+H; autoresizesSubviews = NO; layer = <CALayer: 0x282e6dae0>>
Aug 30 10:02:00 BSC-Connect-Mobile[4741] <Notice>:    |    |    | <GlassWindow: 0x154331450; frame = (0 0; 390 844); autoresizesSubviews = NO; layer = <CALayer: 0x282e12780>>
Aug 30 10:02:00 BSC-Connect-Mobile[4741] <Notice>:    |    |    |    | <UIView: 0x154331af0; frame = (0 0; 390 844); autoresizesSubviews = NO; layer = <CALayer: 0x282e12660>>
Aug 30 10:02:00 BSC-Connect-Mobile[4741] <Notice>:    |    |    |    |    | <GlassViewGL: 0x15485fe00; baseClass = UIScrollView; frame = (0 0; 390 844); clipsToBounds = YES; opaque = NO; gestureRecognizers = <NSArray: 0x28202d620>; layer = <CAEAGLLayer: 0x28202c900>; contentOffset: {585, 1266}; contentSize: {1560, 3376}; adjustedContentInset: {47, 0, 34, 0}>

Notice the 1 before tree/self.view log. I put that in to distinguish between the logging results (before/after capture started). It seems like the completion part is never called. Because nowhere in the log do I find the 2 self.view: , 2 tree view: line.

And this is for iPhone 11 Pro (iOS 15.3):

Aug 30 10:19:50 BSC-Connect-Mobile[3007] <Notice>: 1 self.view:
Aug 30 10:19:50 BSC-Connect-Mobile[3007] <Notice>: <UIView: 0x10bf75650; frame = (0 0; 375 812); autoresize = W+H; layer = <CALayer: 0x283f32500>>
Aug 30 10:19:50 BSC-Connect-Mobile[3007] <Notice>: 1 tree view:
Aug 30 10:19:50 BSC-Connect-Mobile[3007] <Notice>: <UITransitionView: 0x10bf2f750; frame = (0 0; 375 812); autoresize = W+H; layer = <CALayer: 0x283f2b9a0>>
Aug 30 10:19:50 BSC-Connect-Mobile[3007] <Notice>:    | <UIDropShadowView: 0x10bf2fa70; frame = (0 0; 375 812); autoresize = W+H; layer = <CALayer: 0x283f2bb40>>
Aug 30 10:19:50 BSC-Connect-Mobile[3007] <Notice>:    |    | <GlassMainView: 0x10bf2f070; frame = (0 0; 375 812); autoresize = W+H; autoresizesSubviews = NO; layer = <CALayer: 0x283f2b8e0>>
Aug 30 10:19:50 BSC-Connect-Mobile[3007] <Notice>:    |    |    | <GlassWindow: 0x10bf2fc90; frame = (0 0; 375 812); autoresizesSubviews = NO; layer = <CALayer: 0x283f54420>>
Aug 30 10:19:50 BSC-Connect-Mobile[3007] <Notice>:    |    |    |    | <UIView: 0x10bf30080; frame = (0 0; 375 812); autoresizesSubviews = NO; layer = <CALayer: 0x283f54000>>
Aug 30 10:19:50 BSC-Connect-Mobile[3007] <Notice>:    |    |    |    |    | <GlassViewGL: 0x10c080600; baseClass = UIScrollView; frame = (0 0; 375 812); clipsToBounds = YES; opaque = NO; gestureRecognizers = <NSArray: 0x2831381e0>; layer = <CAEAGLLayer: 0x2831380f0>; contentOffset: {562.66666666666663, 1218}; contentSize: {1500, 3248}; adjustedContentInset: {44, 0, 34, 0}>

Here I get the second loggings:

Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>: 2 Completed presenting presentViewController
Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>: 2 self.view:
Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>: <UIView: 0x10bf75650; frame = (0 0; 375 758); autoresize = W+H; layer = <CALayer: 0x283f32500>>
Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>:    | <UINavigationBar: 0x10c87a740; frame = (0 44; 375 44); opaque = NO; layer = <CALayer: 0x283f16be0>> no-scroll-edge-support
Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>:    |    | <_UIBarBackground: 0x10c87ab90; frame = (0 0; 375 56); userInteractionEnabled = NO; layer = <CALayer: 0x283f16a60>>
Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>:    |    |    | <UIImageView: 0x10c87a990; frame = (0 0; 375 56); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0x283f07a00>>
Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>:    |    |    | <_UIBarBackgroundShadowView: 0x10c87ad70; frame = (0 56; 375 0); layer = <CALayer: 0x283f07640>> clientRequestedContentView effect=none
Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>:    |    |    |    | <_UIBarBackgroundShadowContentImageView: 0x10c87e560; frame = (0 0; 375 0); opaque = NO; autoresize = W+H; userInteractionEnabled = NO; layer = <CALayer: 0x283f07480>>
Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>:    |    | <_UINavigationBarContentView: 0x10c87af70; frame = (0 0; 375 56); layer = <CALayer: 0x283f17260>> layout=0x10c87b500
Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>:    |    |    | <_UIButtonBarStackView: 0x10bf99c00; frame =
Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>: 2 tree view:
Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>: <UITransitionView: 0x10bf2f750; frame = (0 0; 375 812); autoresize = W+H; layer = <CALayer: 0x283f2b9a0>>
Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>:    | <UIDropShadowView: 0x10bf2fa70; frame = (16 44; 343 742.709); transform = [0.91466666666666663, 0, 0, 0.91466666666666663, 0, 9.3546666666666738]; clipsToBounds = YES; autoresize = W+H; layer = <CALayer: 0x283f2bb40>>
Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>:    |    | <GlassMainView: 0x10bf2f070; frame = (0 0; 375 812); autoresize = W+H; autoresizesSubviews = NO; layer = <CALayer: 0x283f2b8e0>>
Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>:    |    |    | <GlassWindow: 0x10bf2fc90; frame = (0 0; 375 812); autoresizesSubviews = NO; layer = <CALayer: 0x283f54420>>
Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>:    |    |    |    | <UIView: 0x10bf30080; frame = (0 0; 375 812); autoresizesSubviews = NO; layer = <CALayer: 0x283f54000>>
Aug 30 10:19:51 BSC-Connect-Mobile[3007] <Notice>:    |    |    |    |    | <GlassViewGL: 0x10c080600; baseClass = UIScrollView; frame = (0 0; 375 812); clipsToBounds = YES; opaque = NO; gestureRecognizers = <NSArray: 0x2831381e0>; layer = <CAEAGLLayer: 0x2831380f0>; contentOffset: {562.66666666666663, 1218}; contentSize: {1500, 3248}; adjustedCont

EDIT: I forgot to mention, the iPhone 11 is the one that is working as expected (The camera preview window shows up).

molbertz commented 2 years ago

One more where it's also working. iPhone X (iOS 11.3)

Aug 30 10:37:46 iPhone BSC-Connect-Mobile[2453] <Notice>: 1 self.view:
Aug 30 10:37:46 iPhone BSC-Connect-Mobile[2453] <Notice>: <UIView: 0x107b040f0; frame = (0 0; 375 812); autoresize = W+H; layer = <CALayer: 0x1c4021300>>
Aug 30 10:37:46 iPhone BSC-Connect-Mobile[2453] <Notice>: 1 tree view:
Aug 30 10:37:46 iPhone BSC-Connect-Mobile[2453] <Notice>: <GlassMainView: 0x109d51520; frame = (0 0; 375 812); autoresize = W+H; autoresizesSubviews = NO; layer = <CALayer: 0x1d023f940>>
Aug 30 10:37:46 iPhone BSC-Connect-Mobile[2453] <Notice>:    | <GlassWindow: 0x109e23b40; frame = (0 0; 375 812); autoresizesSubviews = NO; layer = <CALayer: 0x1d4030500>>
Aug 30 10:37:46 iPhone BSC-Connect-Mobile[2453] <Notice>:    |    | <UIView: 0x109e23da0; frame = (0 0; 375 812); autoresizesSubviews = NO; layer = <CALayer: 0x1d4030520>>
Aug 30 10:37:46 iPhone BSC-Connect-Mobile[2453] <Notice>:    |    |    | <GlassViewGL: 0x10d862a00; baseClass = UIScrollView; frame = (0 0; 375 812); clipsToBounds = YES; opaque = NO; gestureRecognizers = <NSArray: 0x1d4056830>; layer = <CAEAGLLayer: 0x1d4029600>; contentOffset: {562.66666666666663, 1218}; contentSize: {1500, 3248}; adjustedContentInset: {44, 0, 34, 0}>

Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>: 2 Completed presenting presentViewController
Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>: 2 self.view:
Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>: <UIView: 0x107b040f0; frame = (0 0; 375 812); autoresize = W+H; layer = <CALayer: 0x1c4021300>>
Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>:    | <UINavigationBar: 0x109e5d470; frame = (0 44; 375 44); opaque = NO; layer = <CALayer: 0x1d4031d20>>
Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>:    |    | <_UIBarBackground: 0x109e5d930; frame = (0 0; 375 44); userInteractionEnabled = NO; layer = <CALayer: 0x1d4031de0>>
Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>:    |    |    | <UIImageView: 0x109e5dd90; frame = (0 0; 375 44); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0x1d40320e0>> - (null)
Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>:    |    |    | <UIImageView: 0x109e5e1c0; frame = (0 44; 375 0); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0x1d4032100>> - (null)
Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>:    |    | <_UINavigationBarLargeTitleView: 0x109e5f310; frame = (0 0; 375 44); clipsToBounds = YES; hidden = YES; layer = <CALayer: 0x1d4032000>>
Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>:    |    |    | <UILabel: 0x109e5fc50; frame = (8.08566 21.3333; 81.162 0); transform = [1.0273672687465791, 0, 0, 1.0273672687465791, 0, 0]; userInteractionEnabled = NO; layer = <_UILabelLayer: 0x1d42814a0>>
Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>:    |    | <_UINavigationBarContentView
Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>: 2 tree view:
Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>: <GlassMainView: 0x109d51520; frame = (0 0; 375 812); autoresize = W+H; autoresizesSubviews = NO; layer = <CALayer: 0x1d023f940>>
Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>:    | <GlassWindow: 0x109e23b40; frame = (0 0; 375 812); autoresizesSubviews = NO; layer = <CALayer: 0x1d4030500>>
Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>:    |    | <UIView: 0x109e23da0; frame = (0 0; 375 812); autoresizesSubviews = NO; layer = <CALayer: 0x1d4030520>>
Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>:    |    |    | <GlassViewGL: 0x10d862a00; baseClass = UIScrollView; frame = (0 0; 375 812); clipsToBounds = YES; opaque = NO; gestureRecognizers = <NSArray: 0x1d4056830>; layer = <CAEAGLLayer: 0x1d4029600>; contentOffset: {562.66666666666663, 1218}; contentSize: {1500, 3248}; adjustedContentInset: {44, 0, 34, 0}>
Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>:    | <AVCaptureVideoPreviewLayer: 0x1d4032880> (layer)
Aug 30 10:37:47 iPhone BSC-Connect-Mobile[2453] <Notice>:    |    | <CALayer: 0x1d4032640> (layer)

jperedadnr commented 2 years ago

Great, that is something!

I've seen somewhere else that [_session startRunning]; might be blocking the UI, so that would explain why the controller doesn't get to complete.

We could try now:

- [_session startRunning];
+ dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
+        NSLog(@"Session start...");
+        [_session startRunning];
+    });

This still works for me (iPhoneX):

2022-08-30 12:52:33.969260+0200 DevoxxBadges[46580:3122894] Completed presenting display
2022-08-30 12:52:33.969287+0200 DevoxxBadges[46580:3123575] Session start...
2022-08-30 12:52:34.522709+0200 DevoxxBadges[46580:3122894] Completed presenting presentViewController
2022-08-30 12:52:34.529614+0200 DevoxxBadges[46580:3122894] self.view:
...

FYI: Hint to speed up iterations: when you only change native code in Attach, you just need to copy the library into your testing project:

$ ./gradlew barcode-scan:publishToMavenLocal
$ cp modules/barcode-scan/build/native/ios/libBarcodeScan.a /path-to-your-project/target/gluonfx/arm64-ios/gvm/lib

then on your project you just skip building the native image all over again (gluonfx:compile) and just do:

$ mvn -Pios gluonfx:link gluonfx:package gluonfx:install gluonfx:nativerun

(as the libBarcodeScan.a library is only added to the link phase).

This will save you some time and CPU usage!

molbertz commented 2 years ago

Thanks for the compile hint.

Still the same issue on iPhone 13 I fear. The dispatch async gets called; the session starts that way. But still the completion handler of the view controller never gets called.

molbertz commented 2 years ago

Interesting. I just send the app to the background after having it inf the "frozen" view for a almost 5 minutes. When I did that the following loggings appeared:

Aug 30 11:31:15 BSC-Connect-Mobile[1473] <Notice>: Completed presenting display
Aug 30 11:31:15 BSC-Connect-Mobile[1473] <Notice>: Session start...
Aug 30 11:31:15 BSC-Connect-Mobile(AVFCapture)[1473] <Notice>: <<<< AVCaptureFigVideoDevice >>>> -[AVCaptureFigVideoDevice _setActiveVideoMinFrameDurationInternal:]: MinFrameDuration to set 1 / 30
Aug 30 11:31:15 BSC-Connect-Mobile(AVFCapture)[1473] <Notice>: <<<< AVCaptureFigVideoDevice >>>> -[AVCaptureFigVideoDevice _setActiveVideoMaxFrameDurationInternal:]: MaxFrameDuration to set 1 / 30
Aug 30 11:31:15 BSC-Connect-Mobile(AVFCapture)[1473] <Notice>:  MOC 0x1060f0620: <SRC:wide back 420v/1920x1080(3), 30-30(max:30), Z:1.00, ICM:0, (FD E:0 B:0 S:0), HR:1, FaceDrivenAEAFMode:0, FaceDrivenAEAFEnabledByDefault:0> -> <SINK 0x10602edb0:MetadataObject> E:1 MetadataIdentifiers:24 rectOfInterest:{{0.00,0.00}{1.00x1.00}} maxFaces:0 usesFaceRecognition:1 faceTrackingPlusEnabled:0
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: sceneOfRecord: sceneID: sceneID:com.bscgmbh.BSCMobileV3-default  persistentID: 41C19297-4271-45FB-A3F8-768AC22C96C4
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: Deactivation reason added: 5; deactivation reasons: 0 -> 32; animating application lifecycle event: 1
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: Deactivation reason added: 12; deactivation reasons: 32 -> 4128; animating application lifecycle event: 1
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: sceneOfRecord: sceneID: sceneID:com.bscgmbh.BSCMobileV3-default  persistentID: 41C19297-4271-45FB-A3F8-768AC22C96C4
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: Deactivation reason added: 5; deactivation reasons: 0 -> 32; animating application lifecycle event: 1
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: Deactivation reason added: 12; deactivation reasons: 32 -> 4128; animating application lifecycle event: 1
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: sceneOfRecord: sceneID: sceneID:com.bscgmbh.BSCMobileV3-default  persistentID: 41C19297-4271-45FB-A3F8-768AC22C96C4
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: sceneOfRecord: sceneID: sceneID:com.bscgmbh.BSCMobileV3-default  persistentID: 41C19297-4271-45FB-A3F8-768AC22C96C4
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: sceneOfRecord: sceneID: sceneID:com.bscgmbh.BSCMobileV3-default  persistentID: 41C19297-4271-45FB-A3F8-768AC22C96C4
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: Deactivation reason added: 11; deactivation reasons: 4128 -> 6176; animating application lifecycle event: 0
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: Sending UIEvent type: 0; subtype: 0; to windows: 1
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: Sending UIEvent type: 0; subtype: 0; to window: <GlassMainWindow: 0x10e92f460>; contextId: 0xF9958982
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: sceneOfRecord: sceneID: sceneID:com.bscgmbh.BSCMobileV3-default  persistentID: 41C19297-4271-45FB-A3F8-768AC22C96C4
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: Deactivation reason added: 11; deactivation reasons: 4128 -> 6176; animating application lifecycle event: 0
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: Sending UIEvent type: 0; subtype: 0; to windows: 1
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: Sending UIEvent type: 0; subtype: 0; to window: <GlassMainWindow: 0x10e92f460>; contextId: 0xF9958982
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: Deactivation reason removed: 5; deactivation reasons: 6176 -> 6144; animating application lifecycle event: 0
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>:   <RBSLegacyAttribute| requestedReason:FinishTask reason:FinishTask flags:( PreventTaskSuspend )>,
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>:   <RBSAcquisitionCompletionAttribute| policy:AfterValidation>
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>:   ]>
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: Deactivation reason removed: 5; deactivation reasons: 6176 -> 6144; animating application lifecycle event: 0
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>:   <RBSLegacyAttribute| requestedReason:FinishTask reason:FinishTask flags:( PreventTaskSuspend )>,
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>:   <RBSAcquisitionCompletionAttribute| policy:AfterValidation>
Aug 30 11:36:19 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>:   ]>
Aug 30 11:36:20 BSC-Connect-Mobile(FrontBoardServices)[1473] <Notice>: Performing snapshot request 0x2804751a0 (type 1)
Aug 30 11:36:20 BSC-Connect-Mobile(FrontBoardServices)[1473] <Notice>: Performing snapshot request 0x2804751a0 (type 1)
Aug 30 11:36:20 BSC-Connect-Mobile(FrontBoardServices)[1473] <Notice>: Snapshot request 0x2804751a0 complete
Aug 30 11:36:20 BSC-Connect-Mobile(FrontBoardServices)[1473] <Notice>: Performing snapshot request 0x2804868e0 (type 1)
Aug 30 11:36:20 BSC-Connect-Mobile(FrontBoardServices)[1473] <Notice>: Snapshot request 0x2804751a0 complete
Aug 30 11:36:20 BSC-Connect-Mobile(FrontBoardServices)[1473] <Notice>: Performing snapshot request 0x2804868e0 (type 1)
Aug 30 11:36:20 BSC-Connect-Mobile(FrontBoardServices)[1473] <Notice>: Snapshot request 0x2804868e0 complete
Aug 30 11:36:20 BSC-Connect-Mobile(FrontBoardServices)[1473] <Notice>: Snapshot request 0x2804868e0 complete
Aug 30 11:36:20 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: [0x2821dce00] [keyboardFocus] Disabling event deferring records requested: adding recreation reason: detachedContext; for reason: _UIEventDeferringManager: 0x2821dce00: disabling keyboardFocus: context detached for window: 0x10e92f460; contextID: 0xF9958982
Aug 30 11:36:20 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: [0x2821dce00] [keyboardFocus] Disabling event deferring records requested: adding recreation reason: detachedContext; for reason: _UIEventDeferringManager: 0x2821dce00: disabling keyboardFocus: context detached for window: 0x10e92f460; contextID: 0xF9958982
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>: 2 Completed presenting presentViewController
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>: 2 self.view:
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>: <UIView: 0x10c107610; frame = (0 0; 390 787); autoresize = W+H; layer = <CALayer: 0x280a84560>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    | <UINavigationBar: 0x10ea78d30; frame = (0 47; 390 44); opaque = NO; layer = <CALayer: 0x280a83c20>> no-scroll-edge-support
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    | <_UIBarBackground: 0x10ea79180; frame = (0 0; 390 56); userInteractionEnabled = NO; layer = <CALayer: 0x280a910e0>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    |    | <UIImageView: 0x10c214dd0; frame = (0 0; 390 56); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0x280a8b8a0>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    |    | <_UIBarBackgroundShadowView: 0x10c215c60; frame = (0 56; 390 0); layer = <CALayer: 0x280a8b940>> clientRequestedContentView effect=none
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    |    |    | <_UIBarBackgroundShadowContentImageView: 0x10c6211e0; frame = (0 0; 390 0); opaque = NO; autoresize = W+H; userInteractionEnabled = NO; layer = <CALayer: 0x280ab4880>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    | <_UINavigationBarContentView: 0x10ea798f0; frame = (0 0; 390 56); layer = <CALayer: 0x280af4200>> layout=0x10ea79b80
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    |    | <_UIButtonBarStackView: 0x10c214c30; frame =
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>: 2 tree view:
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>: <UITransitionView: 0x10c4053c0; frame = (0 0; 390 844); autoresize = W+H; layer = <CALayer: 0x280a9d560>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    | <UIDropShadowView: 0x10e92fc00; frame = (16 47; 358 774.749); transform = [0.91794871794871791, 0, 0, 0.91794871794871791, 0, 12.374358974358984]; clipsToBounds = YES; autoresize = W+H; layer = <CALayer: 0x280a9d580>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    | <GlassMainView: 0x10ea1f380; frame = (0 0; 390 844); autoresize = W+H; autoresizesSubviews = NO; layer = <CALayer: 0x280a98f60>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    |    | <GlassWindow: 0x10ea204f0; frame = (0 0; 390 844); autoresizesSubviews = NO; layer = <CALayer: 0x280a9aa40>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    |    |    | <UIView: 0x10ea20b70; frame = (0 0; 390 844); autoresizesSubviews = NO; layer = <CALayer: 0x280a9a980>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    |    |    |    | <GlassViewGL: 0x11205a200; baseClass = UIScrollView; frame = (0 0; 390 844); clipsToBounds = YES; opaque = NO; gestureRecognizers = <NSArray: 0x2804dda40>; layer = <CAEAGLLayer: 0x2804ddfb0>; contentOffset: {585, 1266}; contentSize: {1560, 3376}; adjustedContentInset: {47,
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>: 2 Completed presenting presentViewController
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>: 2 self.view:
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>: <UIView: 0x10c107610; frame = (0 0; 390 787); autoresize = W+H; layer = <CALayer: 0x280a84560>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    | <UINavigationBar: 0x10ea78d30; frame = (0 47; 390 44); opaque = NO; layer = <CALayer: 0x280a83c20>> no-scroll-edge-support
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    | <_UIBarBackground: 0x10ea79180; frame = (0 0; 390 56); userInteractionEnabled = NO; layer = <CALayer: 0x280a910e0>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    |    | <UIImageView: 0x10c214dd0; frame = (0 0; 390 56); opaque = NO; userInteractionEnabled = NO; layer = <CALayer: 0x280a8b8a0>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    |    | <_UIBarBackgroundShadowView: 0x10c215c60; frame = (0 56; 390 0); layer = <CALayer: 0x280a8b940>> clientRequestedContentView effect=none
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    |    |    | <_UIBarBackgroundShadowContentImageView: 0x10c6211e0; frame = (0 0; 390 0); opaque = NO; autoresize = W+H; userInteractionEnabled = NO; layer = <CALayer: 0x280ab4880>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    | <_UINavigationBarContentView: 0x10ea798f0; frame = (0 0; 390 56); layer = <CALayer: 0x280af4200>> layout=0x10ea79b80
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    |    | <_UIButtonBarStackView: 0x10c214c30; frame =
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>: 2 tree view:
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>: <UITransitionView: 0x10c4053c0; frame = (0 0; 390 844); autoresize = W+H; layer = <CALayer: 0x280a9d560>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    | <UIDropShadowView: 0x10e92fc00; frame = (16 47; 358 774.749); transform = [0.91794871794871791, 0, 0, 0.91794871794871791, 0, 12.374358974358984]; clipsToBounds = YES; autoresize = W+H; layer = <CALayer: 0x280a9d580>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    | <GlassMainView: 0x10ea1f380; frame = (0 0; 390 844); autoresize = W+H; autoresizesSubviews = NO; layer = <CALayer: 0x280a98f60>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    |    | <GlassWindow: 0x10ea204f0; frame = (0 0; 390 844); autoresizesSubviews = NO; layer = <CALayer: 0x280a9aa40>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    |    |    | <UIView: 0x10ea20b70; frame = (0 0; 390 844); autoresizesSubviews = NO; layer = <CALayer: 0x280a9a980>>
Aug 30 11:36:20 BSC-Connect-Mobile[1473] <Notice>:    |    |    |    |    | <GlassViewGL: 0x11205a200; baseClass = UIScrollView; frame = (0 0; 390 844); clipsToBounds = YES; opaque = NO; gestureRecognizers = <NSArray: 0x2804dda40>; layer = <CAEAGLLayer: 0x2804ddfb0>; contentOffset: {585, 1266}; contentSize: {1560, 3376}; adjustedContentInset: {47,
Aug 30 11:36:20 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: sceneOfRecord: sceneID: sceneID:com.bscgmbh.BSCMobileV3-default  persistentID: 41C19297-4271-45FB-A3F8-768AC22C96C4
Aug 30 11:36:20 BSC-Connect-Mobile(UIKitCore)[1473] <Notice>: sceneOfRecord: sceneID: sceneID:com.bscgmbh.BSCMobileV3-default  persistentID: 41C19297-4271-45FB-A3F8-768AC22C96C4

jperedadnr commented 2 years ago

Okay, can you try this small change:

- [rootViewController presentViewController:self animated:YES completion:^(void) {  
+ [rootViewController presentViewController:self animated:NO completion:^(void) {

molbertz commented 2 years ago

Same result again...

jperedadnr commented 2 years ago

Ok, one more, let's not start the session until the controller is presented:

    // show view controller
    [rootViewController presentViewController:self animated:YES completion:^(void) {
        NSLog(@"Completed presenting presentViewController");
        NSLog(@"self.view:\n%@", [self.view recursiveDescription]);
        NSLog(@"tree view:\n%@", [view recursiveDescription]);
        dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
            NSLog(@"Session start...");
            [_session startRunning];
        });
    }];

molbertz commented 2 years ago

"Session start..." is now not called anymore. So the completion handler is really not reached. At least until I press the Home button. That somehow triggers the completion handler...

jperedadnr commented 2 years ago

I've added this four methods right after - (void)display:...:


- (void)viewWillAppear:(BOOL)animated {
    NSLog(@"viewWillAppear");
}

- (void)viewDidAppear:(BOOL)animated {
    NSLog(@"viewDidAppear");
}

- (void)viewWillLayoutSubviews {
    NSLog(@"viewWillLayoutSubviews");
}

- (void)viewDidLayoutSubviews {
    NSLog(@"viewDidLayoutSubviews");
}

Logs show:

2022-08-30 14:47:35.538747+0200 DevoxxBadges[47420:3162895] Completed presenting display
2022-08-30 14:47:35.541205+0200 DevoxxBadges[47420:3162895] viewWillAppear
2022-08-30 14:47:35.543234+0200 DevoxxBadges[47420:3162895] viewWillLayoutSubviews
2022-08-30 14:47:35.543289+0200 DevoxxBadges[47420:3162895] viewDidLayoutSubviews
2022-08-30 14:47:35.576718+0200 DevoxxBadges[47420:3162895] viewWillLayoutSubviews
2022-08-30 14:47:35.579147+0200 DevoxxBadges[47420:3162895] viewDidLayoutSubviews
2022-08-30 14:47:36.086190+0200 DevoxxBadges[47420:3162895] viewDidAppear
2022-08-30 14:47:36.086376+0200 DevoxxBadges[47420:3162895] Completed presenting presentViewController
2022-08-30 14:47:36.093720+0200 DevoxxBadges[47420:3162895] self.view:
...
2022-08-30 14:47:36.098786+0200 DevoxxBadges[47420:3162936] Session start...
...

molbertz commented 2 years ago

The only thing that appears is: Completed presenting display and the first self.view. Nothing else.

jperedadnr commented 2 years ago

Ok, can you start commenting out the lines in display related to the AV parts, like?

- (void)display:(NSString *)title legend:(NSString *)legend resultText:(NSString *)resultText
{
    if(![[UIApplication sharedApplication] keyWindow])
    {
        AttachLog(@"key window was nil");
        return;
    }

    // get the root view controller
    UIViewController *rootViewController = [[[UIApplication sharedApplication] keyWindow] rootViewController];
    if(!rootViewController)
    {
        AttachLog(@"rootViewController was nil");
        return;
    }

    resultString = resultText;

    // get the view
//     UIView *view = self.view;
    NSArray *views = [[[UIApplication sharedApplication] keyWindow] subviews];
    UIView *view = views[0]; // UITransitionView:
    NSLog(@"self.view:\n%@", [self.view recursiveDescription]);
    NSLog(@"tree view:\n%@", [view recursiveDescription]);

    _session = [[AVCaptureSession alloc] init];
//     _device = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
//     NSError *error = nil;
// 
//     _input = [AVCaptureDeviceInput deviceInputWithDevice:_device error:&error];
//     if (_input) {
//         [_session addInput:_input];
//     } else {
//         AttachLog(@"Error: %@", error);
//     }

//     _output = [[AVCaptureMetadataOutput alloc] init];
//     [_output setMetadataObjectsDelegate:self queue:dispatch_get_main_queue()];
//     [_session addOutput:_output];

//     _output.metadataObjectTypes = [_output availableMetadataObjectTypes];

//     _prevLayer = [AVCaptureVideoPreviewLayer layerWithSession:_session];
//     _prevLayer.frame = view.bounds;
//     _prevLayer.videoGravity = AVLayerVideoGravityResizeAspectFill;
//     _prevLayer.connection.videoOrientation = [self videoOrientationFromCurrentDeviceOrientation];
//     [view.layer addSublayer:_prevLayer];

//     CGRect sbFrame = [[UIApplication sharedApplication] statusBarFrame];
//     int ofs = sbFrame.size.height;
//     
//     navBar = [[UINavigationBar alloc] initWithFrame:CGRectMake(0, ofs, self.view.frame.size.width, 44)];
//     [navBar setBackgroundImage:[UIImage new] forBarMetrics:UIBarMetricsDefault];
//     navBar.shadowImage = [UIImage new];
//     navBar.translucent = YES;
//     [navBar setTitleTextAttributes:@{NSForegroundColorAttributeName : [UIColor whiteColor]}];
//     
//     currentItem = [[UINavigationItem alloc] init];
//     if ([title length] != 0) {
//         currentItem.title = title;
//     }

//     UIBarButtonItem *leftButton = [[UIBarButtonItem alloc] initWithTitle:@"Cancel" style:UIBarButtonItemStylePlain 
//             target:self action:@selector(cancel:)];
//     currentItem.leftBarButtonItem = leftButton;
// 
//     navBar.items = @[ currentItem ];
//     [self.view addSubview:navBar];

    // show view controller
    [rootViewController presentViewController:self animated:YES completion:^(void) {
        NSLog(@"Completed presenting presentViewController");
        NSLog(@"self.view:\n%@", [self.view recursiveDescription]);
        NSLog(@"tree view:\n%@", [view recursiveDescription]);
        dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
            NSLog(@"Session start...");
            [_session startRunning];
        });
    }];
//     if ([legend length] != 0) {
//         UIAlertController *toast = [UIAlertController alertControllerWithTitle:nil message:legend preferredStyle:UIAlertControllerStyleAlert];
//         [self presentViewController:toast animated:YES completion:nil];
//         
//         dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
//             [NSThread sleepForTimeInterval:2.0f];   
//             dispatch_async(dispatch_get_main_queue(), ^{
//                 [toast dismissViewControllerAnimated:YES completion:nil];
//             });
//         });
//     }
    NSLog(@"Completed presenting display");
}

molbertz commented 2 years ago

Same thing. Neither the "Session start..." nor the "recursiveDescription" loggings from inside the completion block show up.

jperedadnr commented 2 years ago

Okay, let's try the other way around now:

- (void)display:(NSString *)title legend:(NSString *)legend resultText:(NSString *)resultText
{
    if(![[UIApplication sharedApplication] keyWindow])
    {
        AttachLog(@"key window was nil");
        return;
    }

    // get the root view controller
    UIViewController *rootViewController = [[[UIApplication sharedApplication] keyWindow] rootViewController];
    if(!rootViewController)
    {
        AttachLog(@"rootViewController was nil");
        return;
    }

    resultString = resultText;

    // get the view
//     UIView *view = self.view;
    NSArray *views = [[[UIApplication sharedApplication] keyWindow] subviews];
    UIView *view = views[0]; // UITransitionView:
    NSLog(@"self.view:\n%@", [self.view recursiveDescription]);
    NSLog(@"tree view:\n%@", [view recursiveDescription]);

    _session = [[AVCaptureSession alloc] init];
    _device = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
    NSError *error = nil;

    _input = [AVCaptureDeviceInput deviceInputWithDevice:_device error:&error];
    if (_input) {
        [_session addInput:_input];
    } else {
        AttachLog(@"Error: %@", error);
    }

    _output = [[AVCaptureMetadataOutput alloc] init];
    [_output setMetadataObjectsDelegate:self queue:dispatch_get_main_queue()];
    [_session addOutput:_output];

    _output.metadataObjectTypes = [_output availableMetadataObjectTypes];

    _prevLayer = [AVCaptureVideoPreviewLayer layerWithSession:_session];
    _prevLayer.frame = view.bounds;
    _prevLayer.videoGravity = AVLayerVideoGravityResizeAspectFill;
    _prevLayer.connection.videoOrientation = [self videoOrientationFromCurrentDeviceOrientation];
    [view.layer addSublayer:_prevLayer];

    CGRect sbFrame = [[UIApplication sharedApplication] statusBarFrame];
    int ofs = sbFrame.size.height;

    navBar = [[UINavigationBar alloc] initWithFrame:CGRectMake(0, ofs, self.view.frame.size.width, 44)];
    [navBar setBackgroundImage:[UIImage new] forBarMetrics:UIBarMetricsDefault];
    navBar.shadowImage = [UIImage new];
    navBar.translucent = YES;
    [navBar setTitleTextAttributes:@{NSForegroundColorAttributeName : [UIColor whiteColor]}];

    currentItem = [[UINavigationItem alloc] init];
    if ([title length] != 0) {
        currentItem.title = title;
    }

    UIBarButtonItem *leftButton = [[UIBarButtonItem alloc] initWithTitle:@"Cancel" style:UIBarButtonItemStylePlain 
            target:self action:@selector(cancel:)];
    currentItem.leftBarButtonItem = leftButton;

    navBar.items = @[ currentItem ];
    [view addSubview:navBar];

    // show view controller
//     [rootViewController presentViewController:self animated:YES completion:nil];
    [_session startRunning];

    if ([legend length] != 0) {
        UIAlertController *toast = [UIAlertController alertControllerWithTitle:nil message:legend preferredStyle:UIAlertControllerStyleAlert];
        [self presentViewController:toast animated:YES completion:nil];

        dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
            [NSThread sleepForTimeInterval:2.0f];   
            dispatch_async(dispatch_get_main_queue(), ^{
                [toast dismissViewControllerAnimated:YES completion:nil];
            });
        });
    }
    NSLog(@"Completed presenting display");
}

molbertz commented 2 years ago

It looks like before. The screen appears to be "frozen".

jperedadnr commented 2 years ago

Ok, one more.

I've also read that AV code should be called when viewDidLoad, and with iPhoneX I can verify that that happens when we call display and right when we call UIView *view = self.view;, so all works without any required change.

The current version (see below) uses the UIDimmingView concept (so the controller shows up as a popup from bottom to top and not as full view and it can be actually dragged up and down from the top-centre even if there is no visible handle).

This is my log:

2022-08-30 18:05:12.316110+0200 DevoxxBadges[48310:3214768] BarcodeScanInited
2022-08-30 18:05:12.316410+0200 DevoxxBadges[48310:3214768] BarcodeScan::display
2022-08-30 18:05:12.316438+0200 DevoxxBadges[48310:3214768] BarcodeScan::display 0
2022-08-30 18:05:12.316461+0200 DevoxxBadges[48310:3214768] BarcodeScan::display 1
2022-08-30 18:05:12.316835+0200 DevoxxBadges[48310:3214768] viewDidLoad
2022-08-30 18:05:12.317297+0200 DevoxxBadges[48310:3214768] self.view:
<UIView: 0x109d07250; frame = (0 0; 375 812); autoresize = W+H; layer = <CALayer: 0x28336b700>>
2022-08-30 18:05:12.406082+0200 DevoxxBadges[48310:3214768] present view controller...
2022-08-30 18:05:12.414152+0200 DevoxxBadges[48310:3214768] Start running...
2022-08-30 18:05:12.687719+0200 DevoxxBadges[48310:3214768] Completed presenting display
2022-08-30 18:05:12.702543+0200 DevoxxBadges[48310:3214768] viewWillAppear
2022-08-30 18:05:12.731508+0200 DevoxxBadges[48310:3214768] viewWillLayoutSubviews
2022-08-30 18:05:12.731734+0200 DevoxxBadges[48310:3214768] viewDidLayoutSubviews
2022-08-30 18:05:12.784721+0200 DevoxxBadges[48310:3214768] viewWillLayoutSubviews
2022-08-30 18:05:12.789834+0200 DevoxxBadges[48310:3214768] viewDidLayoutSubviews
2022-08-30 18:05:13.300181+0200 DevoxxBadges[48310:3214768] viewDidAppear

Can you try again and post the log?

Full file:

/*
 * Copyright (c) 2016, 2020, Gluon
 *
 * This program is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.

 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL GLUON BE LIABLE FOR ANY
 * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */

#include "BarcodeScan.h"

JNIEnv *env;

JNIEXPORT jint JNICALL
JNI_OnLoad_BarcodeScan(JavaVM *vm, void *reserved)
{
#ifdef JNI_VERSION_1_8
    //min. returned JNI_VERSION required by JDK8 for builtin libraries
    if ((*vm)->GetEnv(vm, (void **)&env, JNI_VERSION_1_8) != JNI_OK) {
        return JNI_VERSION_1_4;
    }
    return JNI_VERSION_1_8;
#else
    return JNI_VERSION_1_4;
#endif
}

static int BarcodeScanInited = 0;

// BarcodeScan
jclass mat_jScanServiceClass;
jmethodID mat_jScanService_setResult = 0;
BarcodeScan *_barcodeScan;

JNIEXPORT void JNICALL Java_com_gluonhq_attach_barcodescan_impl_IOSBarcodeScanService_initBarcodeScan
(JNIEnv *env, jclass jClass)
{
    if (BarcodeScanInited)
    {
        return;
    }
    BarcodeScanInited = 1;
    NSLog(@"BarcodeScanInited");
    mat_jScanServiceClass = (*env)->NewGlobalRef(env, (*env)->FindClass(env, "com/gluonhq/attach/barcodescan/impl/IOSBarcodeScanService"));
    mat_jScanService_setResult = (*env)->GetStaticMethodID(env, mat_jScanServiceClass, "setResult", "(Ljava/lang/String;)V");
}

void sendScanResult(NSString *scanResult) {
    const char *scanChars = [scanResult UTF8String];
    jstring arg = (*env)->NewStringUTF(env, scanChars);
    (*env)->CallStaticVoidMethod(env, mat_jScanServiceClass, mat_jScanService_setResult, arg);
    (*env)->DeleteLocalRef(env, arg);
    AttachLog(@"Finished sending scan result");
}

JNIEXPORT void JNICALL Java_com_gluonhq_attach_barcodescan_impl_IOSBarcodeScanService_startBarcodeScan
(JNIEnv *env, jclass jClass, jstring jTitle, jstring jLegend, jstring jResult)
{

    const jchar *charsTitle = (*env)->GetStringChars(env, jTitle, NULL);
    NSString *sTitle = [NSString stringWithCharacters:(UniChar *)charsTitle length:(*env)->GetStringLength(env, jTitle)];
    (*env)->ReleaseStringChars(env, jTitle, charsTitle);

    const jchar *charsLegend = (*env)->GetStringChars(env, jLegend, NULL);
    NSString *sLegend = [NSString stringWithCharacters:(UniChar *)charsLegend length:(*env)->GetStringLength(env, jLegend)];
    (*env)->ReleaseStringChars(env, jLegend, charsLegend);

    const jchar *charsResult = (*env)->GetStringChars(env, jResult, NULL);
    NSString *sResult = [NSString stringWithCharacters:(UniChar *)charsResult length:(*env)->GetStringLength(env, jResult)];
    (*env)->ReleaseStringChars(env, jResult, charsResult);

    _barcodeScan = [[BarcodeScan alloc] init];
    NSLog(@"BarcodeScan::display");
    [_barcodeScan display:sTitle legend:sLegend resultText:sResult];
    return;
}

@implementation BarcodeScan 

AVCaptureSession *_session;
AVCaptureDevice *_device;
AVCaptureDeviceInput *_input;
AVCaptureMetadataOutput *_output;
AVCaptureVideoPreviewLayer *_prevLayer;
UINavigationItem *currentItem;
UINavigationBar *navBar;
NSString *resultString;

- (void)display:(NSString *)title legend:(NSString *)legend resultText:(NSString *)resultText
{
    NSLog(@"BarcodeScan::display 0");
    if(![[UIApplication sharedApplication] keyWindow])
    {
        AttachLog(@"key window was nil");
        return;
    }

    // get the root view controller
    UIViewController *rootViewController = [[[UIApplication sharedApplication] keyWindow] rootViewController];
    if(!rootViewController)
    {
        AttachLog(@"rootViewController was nil");
        return;
    }

    resultString = resultText;
NSLog(@"BarcodeScan::display 1");
    // get the view
    UIView *view = self.view;
//     NSArray *views = [[[UIApplication sharedApplication] keyWindow] subviews];
//     UIView *view = views[0]; // UITransitionView:
    NSLog(@"self.view:\n%@", [self.view recursiveDescription]);
//     NSLog(@"tree view:\n%@", [view recursiveDescription]);

    _session = [[AVCaptureSession alloc] init];
    _device = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
    NSError *error = nil;

    _input = [AVCaptureDeviceInput deviceInputWithDevice:_device error:&error];
    if (_input) {
        [_session addInput:_input];
    } else {
        AttachLog(@"Error: %@", error);
    }

    _output = [[AVCaptureMetadataOutput alloc] init];
    [_output setMetadataObjectsDelegate:self queue:dispatch_get_main_queue()];
    [_session addOutput:_output];

    _output.metadataObjectTypes = [_output availableMetadataObjectTypes];

    _prevLayer = [AVCaptureVideoPreviewLayer layerWithSession:_session];
    _prevLayer.frame = view.bounds;
    _prevLayer.videoGravity = AVLayerVideoGravityResizeAspectFill;
    _prevLayer.connection.videoOrientation = [self videoOrientationFromCurrentDeviceOrientation];
    [view.layer addSublayer:_prevLayer];

    CGRect sbFrame = [[UIApplication sharedApplication] statusBarFrame];
    int ofs = sbFrame.size.height;

    navBar = [[UINavigationBar alloc] initWithFrame:CGRectMake(0, ofs, self.view.frame.size.width, 44)];
    [navBar setBackgroundImage:[UIImage new] forBarMetrics:UIBarMetricsDefault];
    navBar.shadowImage = [UIImage new];
    navBar.translucent = YES;
    [navBar setTitleTextAttributes:@{NSForegroundColorAttributeName : [UIColor whiteColor]}];

    currentItem = [[UINavigationItem alloc] init];
    if ([title length] != 0) {
        currentItem.title = title;
    }

    UIBarButtonItem *leftButton = [[UIBarButtonItem alloc] initWithTitle:@"Cancel" style:UIBarButtonItemStylePlain 
            target:self action:@selector(cancel:)];
    currentItem.leftBarButtonItem = leftButton;

    navBar.items = @[ currentItem ];
    [view addSubview:navBar];

    // show view controller
    NSLog(@"present view controller...");
    [rootViewController presentViewController:self animated:YES completion:nil];
    NSLog(@"Start running...");
    [_session startRunning];

    if ([legend length] != 0) {
        UIAlertController *toast = [UIAlertController alertControllerWithTitle:nil message:legend preferredStyle:UIAlertControllerStyleAlert];
        [self presentViewController:toast animated:YES completion:nil];

        dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
            [NSThread sleepForTimeInterval:2.0f];   
            dispatch_async(dispatch_get_main_queue(), ^{
                [toast dismissViewControllerAnimated:YES completion:nil];
            });
        });
    }
    NSLog(@"Completed presenting display");
}

- (void)viewDidLoad {
    NSLog(@"viewDidLoad");
}

- (void)viewWillAppear:(BOOL)animated {
    NSLog(@"viewWillAppear");
}

- (void)viewDidAppear:(BOOL)animated {
    NSLog(@"viewDidAppear");
}

- (void)viewWillLayoutSubviews {
    NSLog(@"viewWillLayoutSubviews");
}

- (void)viewDidLayoutSubviews {
    NSLog(@"viewDidLayoutSubviews");
}

// hide barcodeScan preview and view controller
- (IBAction)cancel:(id)sender
{
    AttachLog(@"Scan cancelled");
    NSString *result = nil;
    sendScanResult(result);
    [self end];
}

- (void)end
{
    if([_session isRunning])
    {
        [_session stopRunning];
    }
    [_session removeInput:_input];
    [_session removeOutput:_output];
    [_prevLayer removeFromSuperlayer];
    [currentItem release];
    currentItem = nil;
    [navBar removeFromSuperview];
    [navBar release];
    navBar = nil;
    [self dismissViewControllerAnimated:YES completion:nil];
    _prevLayer = nil;
    _session = nil;
    resultString = nil;
}

// device will / did rotate
- (void)viewWillTransitionToSize:(CGSize)size withTransitionCoordinator:(id<UIViewControllerTransitionCoordinator>)coordinator
{
    [super viewWillTransitionToSize:size withTransitionCoordinator:coordinator];

    // Place code here which is performed before the rotation animation starts.

    [coordinator animateAlongsideTransition:^(id<UIViewControllerTransitionCoordinatorContext> context)
    {
        // Perform this code here during rotation animation

    } completion:^(id<UIViewControllerTransitionCoordinatorContext> context)
    {

        // rotation finished, resize preview layer
        _prevLayer.frame = self.view.bounds;
        // rotate camera based on new orientation
        _prevLayer.connection.videoOrientation = [self videoOrientationFromCurrentDeviceOrientation];

        CGRect sbFrame = [[UIApplication sharedApplication] statusBarFrame];
        int ofs = sbFrame.size.height;
        navBar.frame = CGRectMake(0, ofs, self.view.frame.size.width, 44);

    }];
}

- (AVCaptureVideoOrientation) videoOrientationFromCurrentDeviceOrientation {
    UIDeviceOrientation orientation = [[UIDevice currentDevice] orientation];

    if (orientation == UIDeviceOrientationPortraitUpsideDown)
        return AVCaptureVideoOrientationPortraitUpsideDown;
    else if(orientation == UIInterfaceOrientationPortrait)
         return AVCaptureVideoOrientationPortrait;
    else if(orientation == UIInterfaceOrientationLandscapeLeft)
        return AVCaptureVideoOrientationLandscapeLeft;
    else if(orientation == UIInterfaceOrientationLandscapeRight)
        return AVCaptureVideoOrientationLandscapeRight;

    return AVCaptureVideoOrientationPortrait;
}

- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputMetadataObjects:(NSArray *)metadataObjects fromConnection:(AVCaptureConnection *)connection
{
    NSString *detectionString = nil;
    NSArray *barCodeTypes = @[AVMetadataObjectTypeUPCECode, AVMetadataObjectTypeCode39Code, AVMetadataObjectTypeCode39Mod43Code,
            AVMetadataObjectTypeEAN13Code, AVMetadataObjectTypeEAN8Code, AVMetadataObjectTypeCode93Code, AVMetadataObjectTypeCode128Code,
            AVMetadataObjectTypePDF417Code, AVMetadataObjectTypeQRCode, AVMetadataObjectTypeAztecCode];

    for (AVMetadataObject *metadata in metadataObjects) {
        for (NSString *type in barCodeTypes) {
            if ([metadata.type isEqualToString:type])
            {
                detectionString = [(AVMetadataMachineReadableCodeObject *)metadata stringValue];
                break;
            }
        }

        if (detectionString != nil)
        {
            break;
        }
        else
        {
//             AttachLog(@"String: none");
        }
    }

    if (detectionString != nil)
    {
        AttachLog(@"String: %@", detectionString);
        if ([resultString length] != 0) {
            if([_session isRunning])
            {
                [_session stopRunning];
            }
            [_session removeInput:_input];
            [_session removeOutput:_output];
            UIAlertController *toast =[UIAlertController alertControllerWithTitle:nil 
                message:[NSString stringWithFormat:@"%@: %@",resultString, detectionString] 
                preferredStyle:UIAlertControllerStyleAlert];
            [self presentViewController:toast animated:YES completion:nil];

            int duration = 2; // in seconds
            dispatch_after(dispatch_time(DISPATCH_TIME_NOW, duration * NSEC_PER_SEC), dispatch_get_main_queue(), ^{
                [toast dismissViewControllerAnimated:YES completion:^{
                    sendScanResult(detectionString);
                    [self end];
                }];
            });
        } else {
            sendScanResult(detectionString);
            [self end];
        }
    }

}
@end

molbertz commented 2 years ago

My log looks almost like yours. Except the "viewWill... / viewDid..." messages are just not there.

Aug 31 05:41:53 BSC-Connect-Mobile[10961] <Notice>: BarcodeScanInited
Aug 31 05:41:53 BSC-Connect-Mobile[10961] <Notice>: BarcodeScan::display
Aug 31 05:41:53 BSC-Connect-Mobile[10961] <Notice>: BarcodeScan::display 0
Aug 31 05:41:53 BSC-Connect-Mobile[10961] <Notice>: BarcodeScan::display 1
Aug 31 05:41:53 BSC-Connect-Mobile[10961] <Notice>: viewDidLoad
Aug 31 05:41:53 BSC-Connect-Mobile[10961] <Notice>: self.view:
Aug 31 05:41:53 BSC-Connect-Mobile[10961] <Notice>: <UIView: 0x15e061230; frame = (0 0; 390 844); autoresize = W+H; layer = <CALayer: 0x2829ba020>>
Aug 31 05:41:53 BSC-Connect-Mobile[10961] <Notice>: present view controller...
Aug 31 05:41:53 BSC-Connect-Mobile[10961] <Notice>: Start running...
Aug 31 05:41:53 BSC-Connect-Mobile[10961] <Notice>: Completed presenting display

There is a lot of other messages between those lines too. If we are at a point where those are of interest I could post them but 1000 lines (~300kB)

jperedadnr commented 2 years ago

Can you post that log as an attached file?

molbertz commented 2 years ago

Okay, I trimmed it down a bit. Where ever there's an empty line there were kernel(AppleH13CameraInterface) messages. Quite a lot, too.

filteredLog_no_camera.txt

jperedadnr commented 2 years ago

Okay, I don't see anything meaningful, do you? What is the behaviour now? You don't see the camera view, but the scanning works (if you point to a barcode it should work)? Looks like again an issue with the UIDimmingView not showing up, but the root controller is not freezing now?

You could try next a combination of previous attempts, like adding back the completion listener to the controller, and commenting out different lines related to the AV session. Also moving code to background threads...

molbertz commented 2 years ago

No, nothing of note. You can see the permission dialog being presented and dismissed. But other than that, everything looks normal to me.

We're checking if the code scan works currently. In the meantime, yesterday I tried to simply push an AlertViewController instead of the "self" one. Works on my Demo Xcode project but not in the compiled library when linked to out app. Could it be that using "the library way" we do not have the correct rootViewController to push new viewControllers?

It seems to me, like some modal is blocking new viewControllers being presented. Which gets dismissed when pressing the Home button (completion handler suddenly prints log messages; on reopen camera preview view appears).

jperedadnr commented 2 years ago

Are you using gluonfx:sharedlib?

molbertz commented 2 years ago

No. gluonfx:build gluonfx:package for our app build.

jperedadnr commented 2 years ago

Ok, fine, so then what do you mean by

using "the library way" ?

molbertz commented 2 years ago

"The library way" With that I mean building our gluon app and adding the attach library as usual. (gluonfx:build gluonfx:package)

I also have set up a single view native app in Xcode where I copied the BarcodeScan class to. That works without problems. That is also where I tried to present a UIAlertController instead of the "self". Works in this native app; does not show up in our gluon app.

We also finished testing the scanning using the last full file of BarcodeScan.m you posted. The camera preview view does not show up; however after (blindly) finding the right distance/position the scan comes back positive. The information is filled into the fields in our view and the buttons on it are responsive again. So it really seems to be an issue of the view not being "pushed" far enough to the top. Do you agree?

jperedadnr commented 2 years ago

Yes, absolutely, that's why I was interested in finding out about the UIView hierarchy.

Can you add this again:

[rootViewController presentViewController:self animated:YES completion:^(void) {
        NSLog(@"Completed presenting presentViewController");
        NSLog(@"self.view:\n%@", [self.view recursiveDescription]);
    }];

It should give us info about the UIDimmingView and its position. If that is not showing it, can you try?

[rootViewController presentViewController:self animated:YES completion:^(void) {
   NSLog(@"Completed presenting presentViewController");
   dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
            [NSThread sleepForTimeInterval:2.0f];   
            dispatch_async(dispatch_get_main_queue(), ^{
                 NSLog(@"self.view:\n%@", [self.view recursiveDescription]);
            });
        });
});

and keep the scanner open (without scanning a barcode) for at least 2 seconds?

molbertz commented 2 years ago

Tried this:

[rootViewController presentViewController:self animated:YES completion:^(void) {
        NSLog(@"Completed presenting presentViewController");
        NSLog(@"self.view:\n%@", [self.view recursiveDescription]);
    }];

No logs from completion handler.

Then I also tried this:

[rootViewController presentViewController:self animated:YES completion:^(void) {
   NSLog(@"Completed presenting presentViewController");
   dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
            [NSThread sleepForTimeInterval:2.0f];   
            dispatch_async(dispatch_get_main_queue(), ^{
                 NSLog(@"self.view:\n%@", [self.view recursiveDescription]);
            });
        });
});

Again, no NSLog(@"Completed presenting presentViewController"); from completion handler and therefore no dispatch_async and no NSLog(@"self.view:\n%@", [self.view recursiveDescription]);

jperedadnr commented 2 years ago

Let's see if the keyWindow deprecation has to do with that...

Can you replace the first lines in display with:

    UIWindow *keyWindow = nil;
    for (UIWindow *window in [[UIApplication sharedApplication] windows]) {
        if (window.isKeyWindow) {
            keyWindow = window;
            break;
        }
    }
    if (!keyWindow)
    {
        AttachLog(@"keyWindow was nil");
        return;
    }
    // get the root view controller
    UIViewController *rootViewController = keyWindow.rootViewController;
    if (!rootViewController)
    {
        AttachLog(@"rootViewController was nil");
        return;
    }

and try again?

molbertz commented 2 years ago

Same behavior. But the keyWindow thing seems to work since neither "... was nil" log gets printed.

jperedadnr commented 2 years ago

I've added after last line above:

NSLog(@"first root: %@", rootViewController);
    while (rootViewController.presentedViewController)
    {
        rootViewController = rootViewController.presentedViewController;
    }
    NSLog(@"top root: %@", rootViewController);

but on iPhoneX it is the same:

first root: <GlassViewController: 0x10491b6c0>
top root: <GlassViewController: 0x10491b6c0>

Can you check?

molbertz commented 2 years ago

Same thing on iPhone 13 Pro

Aug 31 12:42:25 BSC-Connect-Mobile[2758] <Notice>: first root: <GlassViewController: 0x10b9391a0>
Aug 31 12:42:25 BSC-Connect-Mobile[2758] <Notice>: top root: <GlassViewController: 0x10b9391a0>

jperedadnr commented 2 years ago

Can you replace the

 NSLog(@"present view controller...");
    [rootViewController presentViewController:self animated:YES completion:...];

with

NSLog(@"present view controller...");
    [rootViewController showViewController:self sender:rootViewController];

jperedadnr commented 2 years ago

Also you can add this log:

- (void)viewDidAppear:(BOOL)animated {
    NSLog(@"viewDidAppear at x= %f y= %f w= %f h=%f", self.view.bounds.origin.x, self.view.bounds.origin.y, self.view.bounds.size.width, self.view.bounds.size.height);
}

On iPhoneX:

viewDidAppear at x= 0.000000 y= 0.000000 w= 375.000000 h=758.000000

molbertz commented 2 years ago

No change. Also, no "viewDidAppear" logging shows up.

jperedadnr commented 2 years ago

Okay, let's try what does work and modify viewDidLoad with:

- (void)viewDidLoad {
    NSLog(@"viewDidLoad at x= %f y= %f w= %f h=%f", self.view.bounds.origin.x, self.view.bounds.origin.y, self.view.bounds.size.width, self.view.bounds.size.height);
    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
        [NSThread sleepForTimeInterval:2.0f];
        dispatch_async(dispatch_get_main_queue(), ^{
            NSLog(@"did load self.view:\n%@", [self.view recursiveDescription]);
        });
    });
}

molbertz commented 2 years ago

Okay, here's what I got. FYI: I'll be off for today. Will continue tomorrow though.

Aug 31 15:38:38 BSC-Connect-Mobile[5174] <Notice>: viewDidLoad at x= 0.000000 y= 0.000000 w= 390.000000 h=844.000000

Aug 31 15:38:40 BSC-Connect-Mobile[5174] <Notice>: did load self.view:
Aug 31 15:38:40 BSC-Connect-Mobile[5174] <Notice>: <UIView: 0x108e5b180; frame = (0 0; 390 844); autoresize = W+H; layer = <CALayer: 0x281015ea0>>
Aug 31 15:38:40 BSC-Connect-Mobile[5174] <Notice>:    | <AVCaptureVideoPreviewLayer: 0x281e31f20> (layer)
Aug 31 15:38:40 BSC-Connect-Mobile[5174] <Notice>:    |    | <CALayer: 0x2810ead20> (layer)
Aug 31 15:38:40 BSC-Connect-Mobile[5174] <Notice>:    | <UINavigationBar: 0x108e858e0; frame = (0 47; 390 44); opaque = NO; layer = <CALayer: 0x2810cfaa0>> no-scroll-edge-support

jperedadnr commented 2 years ago

Great, no worries! thanks for all your help.

On X:

 2022-08-31 17:24:30.372417+0200 DevoxxBadges[52227:3466268] viewDidAppear at x= 0.000000 y= 0.000000 w= 375.000000 h=758.000000
 <UIView: 0x10aa06510; frame = (0 0; 375 758); autoresize = W+H; layer = <CALayer: 0x2812c5e00>>
   | <AVCaptureVideoPreviewLayer: 0x281cd5b00> (layer)
   |    | <CALayer: 0x2812fc480> (layer)
   | <UINavigationBar: 0x108e39e10; frame = (0 44; 375 44); opaque = NO; layer = <CALayer: 0x2812f3d20>> no-scroll-edge-support
   |    | <_UIBarBackground: 0x108e3a260; frame = (0 0; 375 56); userInteractionEnabled = NO; layer = <CALayer: 0x2812f3a60>>
...

molbertz commented 2 years ago

Okay here's what I found out today. I think the problem is not the views/controllers at all. It seems like Platform.enterNestedEventLoop(result); from IOSBarcodeScanService.java is the culprit.

I commented out this and the exitNestedEventLoop line and simply presented a new UIAlertController in JNIEXPORT void JNICALL Java_com_gluonhq_attach_barcodescan_impl_IOSBarcodeScanService_startBarcodeScan of BarcodeScan.m.

Now, the alert view shows up. With the nestedEventLoop lines in place I get the same behavior as before. The view is frozen.

I assume you need these lines so that the scan method is waiting for the scan result on the Java side. I'm afraid I don't know how else to achieve that. But to reiterate, I think for some reason this causes the view to freeze on iPhone 13 Pro but not on iPhone X.

jperedadnr commented 2 years ago

That's a great finding! One easy thing we can do for now is a small API change, at least for testing:

patch.diff.txt

Then in your project just change the service:

BarcodeScanService.create().ifPresentOrElse(s -> {
                s.resultProperty().addListener((obs, ov, nv) -> {
                    if (nv != null) {
                        System.out.println("Got result: " + nv);
                    }
                });
                s.scan();
            },
            () -> addBadge(getDummyQR()));
        });

molbertz commented 2 years ago

I'm trying to build the library but I'm having a lot of trouble. If I just build it like I used to, the app build fails because the new ios service api is not known to our project (keeps insisting there is just the old one where the scan method returns the Optional).

So I tried to just build the entire barcode-scan lib. But even after downloading and the android sdk the androidBuild task still fails. Now with Unrecognized VM option 'MaxPermSize=1924m'

I tried to exclude the android task (because we don't need it yet) but he still keeps executing it.

./gradlew :barcode-scan:clean build --exclude-task :barcode-scan:androidBuild

How can I build just the barcode-scan lib so that our project uses the api changes to the BarcodeScanService.java?