search

apivideo / api.video-reactnative-live-stream

React Native RTMP live stream client. Made with ♥ by api.video
https://api.video
MIT License
176 stars 32 forks source link

[Bug]: Android camera 'back' is used when camera prop is 'front' #81

Open giantslogik opened 3 months ago

giantslogik commented 3 months ago

Version

v2.0.0

Which operating systems have you used?

Environment that reproduces the issue

Google Pixel 7

Is it reproducible in the example application?

Not tested

RTMP Server

NA, happens before streaming

Reproduction steps

  1. pass prop camera={'front'}

    <ApiVideoLiveStreamView
      key={'ApiVideoLiveStreamView'}
      style={styles.liveStreamView}
      ref={ref}
      camera={'front'}
      enablePinchedZoom={true}
      video={{
        fps: 30,
        resolution: { width: 1080, height: 1920 },
        bitrate: 2 * 1024 * 1024, // # 2 Mbps
        gopDuration: 1 // 1 second
      }}
      audio={{
        bitrate: 128000,
        sampleRate: 44100,
        isStereo: true
      }}
      isMuted={false}
      onConnectionSuccess={onConnectionSuccess}
      onConnectionFailed={onConnectionFailed}
      onDisconnect={onDisconnect}
    />

Expected result

'front' camera is used

Actual result

'back' camera is used.

Additional context

The bug report form did not have that version v2.0.0. !!! Works correctly on iOS.

Workaround:

  1. Set camera prop to undefined initially.
  2. Within a setTimeout (500ms - 1sec) trigger state changes to set the camera prop to 'front'.

Relevant logs output

No response

ThibaultBee commented 3 months ago

Hi,

It is a random issue. I made changes in the Android dependency. It will be fixed in 2.0.1.

ThibaultBee commented 2 months ago

Hi, Could you test on main branch?

giantslogik commented 1 month ago

@ThibaultBee , I tried to test main on Android, and it seems like the correct camera is initially used. I could not test it much as the library crashes for me with:

 E  FATAL EXCEPTION: pool-40-thread-1
 Process: com.**********, PID: 2574
 java.lang.IllegalStateException: LifecycleScope is not available
at io.github.thibaultbee.streampack.views.PreviewView.startPreviewIfReady(PreviewView.kt:272)
at io.github.thibaultbee.streampack.views.PreviewView.startPreview(PreviewView.kt:245)
at video.api.livestream.views.ApiVideoView.startPreview$livestream_release(ApiVideoView.kt:62)
at video.api.livestream.ApiVideoLiveStream$startPreview$1.invoke(ApiVideoLiveStream.kt:373)
at video.api.livestream.ApiVideoLiveStream$startPreview$1.invoke(ApiVideoLiveStream.kt:364)
at video.api.reactnative.livestream.LiveStreamView$1$1.invoke(LiveStreamView.kt:75)
at video.api.reactnative.livestream.LiveStreamView$1$1.invoke(LiveStreamView.kt:72)
at video.api.reactnative.livestream.utils.permissions.SerialPermissionsManager$requestPermissions$request$1$1.invoke(SerialPermissionsManager.kt:38)
at video.api.reactnative.livestream.utils.permissions.SerialPermissionsManager$requestPermissions$request$1$1.invoke(SerialPermissionsManager.kt:35)
at video.api.reactnative.livestream.utils.permissions.PermissionsManager$requestPermissions$1.onAllGranted(PermissionsManager.kt:51)
at video.api.reactnative.livestream.utils.permissions.PermissionsManager.requestPermissions(PermissionsManager.kt:76)
at video.api.reactnative.livestream.utils.permissions.PermissionsManager.requestPermissions(PermissionsManager.kt:49)
at video.api.reactnative.livestream.utils.permissions.SerialPermissionsManager.requestPermissions$lambda$1(SerialPermissionsManager.kt:35)
at video.api.reactnative.livestream.utils.permissions.SerialPermissionsManager.$r8$lambda$dzig6asy22NzT_1QrKBawhgieas(Unknown Source:0)
at video.api.reactnative.livestream.utils.permissions.SerialPermissionsManager$$ExternalSyntheticLambda1.run(Unknown Source:10)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
at java.lang.Thread.run(Thread.java:1012)

The scenario it crashes is either (re) Mount or unmount of ApiVideoLiveStreamView. In my flow i have multiple screens displaying ApiVideoLiveStreamView:

  1. Screen with camera selector + next button.
  2. Screen with stream configuration. + next button
  3. Actual live streaming Screen, where users can stream by pressing a "record" button.

Screens 1 & 2 just show the camera output. 3 does the streaming. The crash happens when moving between screen 1 & 2 above. This worked ok with2.0.0+iOS/Android and main+iOS

ThibaultBee commented 1 month ago

@giantslogik Could you provide a sample where I can test directly?

giantslogik commented 1 month ago

I'll try and modify the example in the repo. May take me a while to get to it.

ThibaultBee commented 1 month ago

The exception comes from here: https://github.com/ThibaultBee/StreamPack/blob/079ff7da527d78ed7e60daeeb7ff8bc9015edd6e/core/src/main/java/io/github/thibaultbee/streampack/views/PreviewView.kt#L272 The question is : Why does it happen?

ThibaultBee commented 1 month ago

Instead of throwing an exception here, we can just log an error. But that might hide other issues.

giantslogik commented 1 month ago

This worked without an Exception with api.video-reactnative-live-stream 2.0.0 Android. I'm trying to see what has changed in streampack and api.video-reactnative-live-stream since then. Do you have tags i can diff by. Having some trouble determining what version of streampack is used by api.video-reactnative-live-stream 2.0.0

giantslogik commented 1 month ago

I think that this commit , introduced the issue:

https://github.com/ThibaultBee/StreamPack/commit/27d3a9e8c8563c17139bc14b2e9397dd8df777b8#diff-0e5bf720cab5ddf6afd6d26f19aa08330afc6d2615f80e6609c572eb0aa45cb1

ThibaultBee commented 1 month ago

I know but I want to be sure there isn't any side effect in the patch and if this would solve your issue. Have you made progress on a standalone sample where I can reproduce this issue?

giantslogik commented 1 month ago

I haven't had the time to write a sample. I will pull main again , set shouldFailSilently = true and see if that works. (i.e. try the "Instead of throwing an exception here, we can just log an error." )

ThibaultBee commented 2 weeks ago

@giantslogik another way of dealing with this is to catch the exception when calling startPreview. I could create a branch with this fix. Could you test it?

safee-cases commented 1 week ago

I also have this problem on my application. Only on android.

I'm currently on the main branch because I had to fix another error I had before, which can be found on this issue. #82 (java.lang.NullPointerException: Attempt to invoke virtual method 'int android.media.audiofx.AcousticEchoCanceler.setEnabled(boolean)' on a null object reference)

Now, I got the "LifecycleScope" when I start a livestream, here is my log error:

Your app just crashed. See the error below.
java.lang.IllegalStateException: LifecycleScope is not available
  io.github.thibaultbee.streampack.views.PreviewView.startPreviewIfReady(PreviewView.kt:272)
  io.github.thibaultbee.streampack.views.PreviewView.startPreview(PreviewView.kt:245)
  video.api.livestream.views.ApiVideoView.startPreview$livestream_release(ApiVideoView.kt:62)
  video.api.livestream.ApiVideoLiveStream$startPreview$1.invoke(ApiVideoLiveStream.kt:373)
  video.api.livestream.ApiVideoLiveStream$startPreview$1.invoke(ApiVideoLiveStream.kt:364)
  video.api.reactnative.livestream.LiveStreamView$1$1.invoke(LiveStreamView.kt:75)
  video.api.reactnative.livestream.LiveStreamView$1$1.invoke(LiveStreamView.kt:72)
  video.api.reactnative.livestream.utils.permissions.SerialPermissionsManager$requestPermissions$request$1$1.invoke(SerialPermissionsManager.kt:38)
  video.api.reactnative.livestream.utils.permissions.SerialPermissionsManager$requestPermissions$request$1$1.invoke(SerialPermissionsManager.kt:35)
  video.api.reactnative.livestream.utils.permissions.PermissionsManager$requestPermissions$1.onAllGranted(PermissionsManager.kt:51)
  video.api.reactnative.livestream.utils.permissions.PermissionsManager.requestPermissions(PermissionsManager.kt:76)
  video.api.reactnative.livestream.utils.permissions.PermissionsManager.requestPermissions(PermissionsManager.kt:49)
  video.api.reactnative.livestream.utils.permissions.SerialPermissionsManager.requestPermissions$lambda$1(SerialPermissionsManager.kt:35)
  video.api.reactnative.livestream.utils.permissions.SerialPermissionsManager.$r8$lambda$dzig6asy22NzT_1QrKBawhgieas(Unknown Source:0)
  video.api.reactnative.livestream.utils.permissions.SerialPermissionsManager$$ExternalSyntheticLambda1.run(Unknown Source:10)
  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
  java.lang.Thread.run(Thread.java:1012)

Is the branch with the fix available? I could test it. Thank you !

ThibaultBee commented 1 week ago

@giantslogik @safee-cases Could you test https://github.com/apivideo/api.video-reactnative-live-stream/pull/90? I am trying to find the source of the crash. If you find anything relevant so I can reproduce it, please tell us :)

giantslogik commented 1 week ago

@ThibaultBee I tested with "@api.video/react-native-livestream": "https://github.com/apivideo/api.video-reactnative-live-stream.git#bugfix/missing_lifecycle_scope",

Still crashes.

2024-09-05 15:35:18.916 14990-15818 AndroidRuntime          ***************            E  FATAL EXCEPTION: pool-35-thread-1
                                                                                                    Process: *************, PID: 14990
                                                                                                    java.lang.IllegalStateException: LifecycleScope is not available
                                                                                                        at io.github.thibaultbee.streampack.views.PreviewView.startPreviewIfReady(PreviewView.kt:272)
                                                                                                        at io.github.thibaultbee.streampack.views.PreviewView.startPreview(PreviewView.kt:245)
                                                                                                        at video.api.livestream.views.ApiVideoView.startPreview$livestream_release(ApiVideoView.kt:62)
                                                                                                        at video.api.livestream.ApiVideoLiveStream$startPreview$1.invoke(ApiVideoLiveStream.kt:373)
                                                                                                        at video.api.livestream.ApiVideoLiveStream$startPreview$1.invoke(ApiVideoLiveStream.kt:364)
                                                                                                        at video.api.reactnative.livestream.LiveStreamView$1$1.invoke(LiveStreamView.kt:75)
                                                                                                        at video.api.reactnative.livestream.LiveStreamView$1$1.invoke(LiveStreamView.kt:72)
                                                                                                        at video.api.reactnative.livestream.utils.permissions.SerialPermissionsManager$requestPermissions$request$1$1.invoke(SerialPermissionsManager.kt:38)
                                                                                                        at video.api.reactnative.livestream.utils.permissions.SerialPermissionsManager$requestPermissions$request$1$1.invoke(SerialPermissionsManager.kt:35)
                                                                                                        at video.api.reactnative.livestream.utils.permissions.PermissionsManager$requestPermissions$1.onAllGranted(PermissionsManager.kt:51)
                                                                                                        at video.api.reactnative.livestream.utils.permissions.PermissionsManager.requestPermissions(PermissionsManager.kt:76)
                                                                                                        at video.api.reactnative.livestream.utils.permissions.PermissionsManager.requestPermissions(PermissionsManager.kt:49)
                                                                                                        at video.api.reactnative.livestream.utils.permissions.SerialPermissionsManager.requestPermissions$lambda$1(SerialPermissionsManager.kt:35)
                                                                                                        at video.api.reactnative.livestream.utils.permissions.SerialPermissionsManager.$r8$lambda$dzig6asy22NzT_1QrKBawhgieas(Unknown Source:0)
                                                                                                        at video.api.reactnative.livestream.utils.permissions.SerialPermissionsManager$$ExternalSyntheticLambda1.run(Unknown Source:10)
                                                                                                        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
                                                                                                        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
                                                                                                        at java.lang.Thread.run(Thread.java:1012)

This is a race condition between unmounting previous LiveStreamView and mounting another LiveStreamView. (Since i display it on two consecutive screens.) I was able to avoid the crash by using a wrapper component which implements the following logic. (Don't know if its works consistently, and it causes flicker and flashing of the screen.)

enum LiveVideoStreamInitLifecycle {
  NOT_INITALIZED,
  INITIALIZING,
  INITIALIZED
}

export type LiveVideoStreamProps = {
  camera: 'front' | 'back' | undefined;
  /* Mitigate bugs by a delayed initialize.*/
  delayInit?: boolean;
  ....
};

const LiveVideoStream = forwardRef<LiveVideoStreamMethods, LiveVideoStreamProps>((props, extRef) => {
  let {
    camera,
    delayInit = true,
    ....
  } = props;

  const [initialized, setInitialized] = useState<LiveVideoStreamInitLifecycle>(
    delayInit ? LiveVideoStreamInitLifecycle.NOT_INITALIZED : LiveVideoStreamInitLifecycle.INITIALIZING
  );

  useEffect(() => {
    setTimeout(
      () => {
        setInitialized(
          initialized === LiveVideoStreamInitLifecycle.NOT_INITALIZED
            ? LiveVideoStreamInitLifecycle.INITIALIZING
            : LiveVideoStreamInitLifecycle.INITIALIZED
        );
      },
      initialized === LiveVideoStreamInitLifecycle.NOT_INITALIZED ? 0 : 500
    );
  }, [initialized]);

camera = initialized === LiveVideoStreamInitLifecycle.INITIALIZED ? camera : undefined;

return (
    <>
      {initialized ? (
        <ApiVideoLiveStreamView
          key={'ApiVideoLiveStreamView'}
          style={styles.streamstyle}
          ref={ref}
          camera={camera}
          enablePinchedZoom={true}
          video={{
            fps: 30,
            resolution: { width: 1080, height: 1920 },
            bitrate: 2 * 1024 * 1024, // # 2 Mbps
            gopDuration: 1 // 1 second
          }}
          audio={{
            bitrate: 128000,
            sampleRate: 44100,
            isStereo: true
          }}
          isMuted={false}
.....
        />
      ) : null}
   </>
  );
});
ThibaultBee commented 1 day ago

I still don't understand why it happens... But anyway, as we don't have a way to reproduce it and their is a workaround, I am going to release the 2.0.1 without a fix and we will open an issue with this issue.

ThibaultBee commented 1 day ago

Could you test the original issue on 2.0.1?