search

RenderHeads / UnityPlugin-AVProVideo

AVPro Video is a multi-platform Unity plugin for advanced video playback
https://www.renderheads.com/products/avpro-video/
238 stars 29 forks source link

Destroying MediaPlayer Gameobject stuck the main thread for over 20 seconds / Dolby not supported in ExoPlayer #1302

Closed zyfeng413 closed 1 year ago

zyfeng413 commented 2 years ago

When the network video is opened without network or the network is interrupted in the middle of playing the network video, the main thread will be stuck for a long time when quitting the playing, The m_Video.Call("CloseVideo") interface was found to be stuck.How can I solve this problem? Thank you！

environment： Unity 2020.3.30 Platform： Android

Ste-RH commented 2 years ago

What version of AVPro Video are you using?

What API Path are you using?

Which device(s) have you seen this on?

Can you send over a full logcat please?

Ste-RH commented 2 years ago

@zyfeng413 On Android, the MediaPlayer API opens/closes videos on a thread, and so should not hit the main thread much/at-all. This is unfortunately something we cannot do in the ExoPlayer API path currently. So, if you are not already using the MediPlayer API path, and can, it might be worth trying that.

zyfeng413 commented 2 years ago

@Ste-RH AVPro Video - Ultra Edition v2.6.2 API Path is MediaPlayer, the devices is Car machine SOC, I'll try a few more devices recurrence： Call mediaPlayer.openMedia without a network, then destroy MediaPlayer.gameObject

Ste-RH commented 2 years ago

I would also try using ExoPlayer to see if it is different

zyfeng413 commented 2 years ago

I see this log from main.txt W m.xiaopeng.nap: Long monitor contention with owner Binder:3037_1 (3066) at java.net.InetAddress[] libcore.io.Linux.android_getaddrinfo(java.lang.String, android.system.StructAddrinfo, int)(Linux.java:-2) waiters=0 in void android.media.MediaHTTPConnection.disconnect() for 26.129s I hope it helps with the analysis.I also tried two other mobile devices (HarmonyOS 2.0.0 and Android 8.1.0), but did not reproduce the problem. Currently available only on Car Machine SOC (Android 10).

Ste-RH commented 2 years ago

Can you provide a link to the device? I am struggling to find it.

When does the pause happen? During the GO destroy? Could you try to wind it back a little more sensibly? (wait for the open to fail, then call Dispose() on the media player before the GO destroy)

It does feel like a driver or OS level issue with that device though - as you, us, other developers, are not seeing this issue on other devices. The IO logging you point at is not called directly by us in AVPro Video c# or plugin code - that is not to say we are not calling API level functions that do not call them. The MediaPlayer API path implementation in AVPro Video is threaded and so should not hit the main thread at all.

zyfeng413 commented 2 years ago

I'm sorry that I can't provide the device link at present, because our company's car is still in the unmarketed stage. I can provide more logs to help with analysis.

"UnityMain" prio=5 tid=24 Blocked | group="main" sCount=1 dsCount=0 flags=1 obj=0x155c0ed8 self=0x7ae5d31400 | sysTid=20649 nice=0 cgrp=default sched=0/0 handle=0x7a85a74d50 | state=S schedstat=( 58827471983 565784310 38244 ) utm=5401 stm=481 core=7 HZ=100 | stack=0x7a8596a000-0x7a8596c000 stackSize= 1071KB pos = 0x1488aeb0 thread_local_limit = 0x148c0000 thread_local_objects = 90 | held mutexes= at android.media.MediaHTTPConnection.disconnect(MediaHTTPConnection.java:171)

"Binder:20618_8" prio=5 tid=162 Native | group="main" sCount=1 dsCount=0 flags=1 obj=0x132c1310 self=0x78e99d2800 | sysTid=20987 nice=0 cgrp=default sched=0/0 handle=0x7914382d50 | state=S schedstat=( 577469522 307989946 1534 ) utm=55 stm=2 core=3 HZ=100 | stack=0x7914284000-0x7914286000 stackSize= 1023KB pos = 0x0 thread_local_limit = 0x0 thread_local_objects = 0 | held mutexes= kernel: (couldn't read /proc/self/task/20987/stack) native: #00 pc 00000000000cd5e4 /apex/com.android.runtime/lib64/bionic/libc.so (read+4) native: #01 pc 00000000000d8e40 /apex/com.android.runtime/lib64/bionic/libc.so (sread+40) native: #02 pc 00000000000d8d3c /apex/com.android.runtime/lib64/bionic/libc.so (srefill+260) native: #03 pc 00000000000dce74 /apex/com.android.runtime/lib64/bionic/libc.so (fread_unlocked+204) native: #04 pc 00000000000dcd64 /apex/com.android.runtime/lib64/bionic/libc.so (fread+72) native: #05 pc 0000000000098ec0 /apex/com.android.runtime/lib64/bionic/libc.so (android_getaddrinfofornetcontext+2396) native: #06 pc 000000000009853c /apex/com.android.runtime/lib64/bionic/libc.so (android_getaddrinfofornet+56) native: #07 pc 000000000002c6f8 /apex/com.android.runtime/lib64/libjavacore.so (Linux_android_getaddrinfo(_JNIEnv, _jobject, _jstring, _jobject, int)+336) at libcore.io.Linux.android_getaddrinfo(Native method) at libcore.io.ForwardingOs.android_getaddrinfo(ForwardingOs.java:74) at libcore.io.BlockGuardOs.android_getaddrinfo(BlockGuardOs.java:200) at libcore.io.ForwardingOs.android_getaddrinfo(ForwardingOs.java:74) at java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:135) at java.net.Inet6AddressImpl.lookupAllHostAddr(Inet6AddressImpl.java:103) at java.net.InetAddress.getAllByName(InetAddress.java:1152) at com.android.okhttp.Dns$1.lookup(Dns.java:41) at com.android.okhttp.internal.http.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:178) at com.android.okhttp.internal.http.RouteSelector.nextProxy(RouteSelector.java:144) at com.android.okhttp.internal.http.RouteSelector.next(RouteSelector.java:86) at com.android.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:176) at com.android.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:128) at com.android.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:97) at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:289) at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:232) at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:465) at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:411) at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:542) at android.media.MediaHTTPConnection.seekTo(MediaHTTPConnection.java:284)

Ste-RH commented 2 years ago

It looks like something has called 'seek' (not evident as to what this is as the callstack does not show enough information, and that seek is locking the main thread? Because the 'close' calls the MediaPlayer 'reset()' on a Runnable/Thread model.

I guess we are a bit stuck as the device you are having issue with is not available to us in any form! Even with the device in hand we could not promise to be able to address the issue. Though, it does appear to be limited to that device specifically? Is there anything 'special' about the version of Android on the device? Or the hardware used in the device?

Chris-RH commented 2 years ago

@zyfeng413

Is there any further information that you could provide? Is there anything 'special' about the version of Android on the device? Or the hardware used in the device?

zyfeng413 commented 2 years ago

@Ste-RH @Chris-RH What's special about this device is that even if I turn off wifi and 4G, there's a built-in weak network that the system doesn't consider offline

zyfeng413 commented 2 years ago

@Ste-RH @Chris-RH Whether we can set the timeout period through the interface or other means?

Ste-RH commented 2 years ago

Any timeout is beyond the scope of AVPro Video as (both) the network calls are happening deep inside the Android SDK

zyfeng413 commented 2 years ago

This bothered me so much that I tried switching the API Path to ExoPlayer, which won't stuck the main thread, but unfortunately doesn't support playing our Dolby videos. But I can play it using the official exoplayer demo.

Ste-RH commented 2 years ago

We will take a look to see if we can add support for the format you are using

Ste-RH commented 2 years ago

The ExoPlayer demo app reports back that 'audio tracks are present but not supported by this device'.

Track info logged out is:

EventLogger: tracks [eventTime=0.36, mediaPos=0.00, window=0, period=0 EventLogger: group EventLogger: [ EventLogger: [X] Track:0, id=1, mimeType=video/avc, codecs=avc1.640032, res=2400x1200, fps=25.0, supported=YES EventLogger: ] EventLogger: group EventLogger: [ EventLogger: [ ] Track:0, id=2, mimeType=audio/eac3-joc, channels=6, sample_rate=48000, language=und, supported=NO_UNSUPPORTED_TYPE EventLogger: ] EventLogger: ]

I am using a Xiaomi Mi 10T

There is no reason I can see that, if the device supports eac3 then both ExoPlayer demo and AVPro Video would not play it back.

I will try some more devices I have in hand to see if any of them support eac3

Ste-RH commented 2 years ago

The only other device I have to hand the might play it is an Oculus Quest 2, and it does not - either in the ExoPlayer demo app or AVPro Video. Again, it says the audio track is unsupported.

Maybe you could send over a full logcat of yo running the ExoPlayer demo app? You can email it over to unitysupport@renderheads.com if you wish to keep it private.

zyfeng413 commented 2 years ago

I have consulted our colleagues in the system department, our system is Android10, and there are two modifications to the ExoPlayer demo to support audio. Here are the full logcat and code changes that play normally. log.txt img_v2_1978f788-079c-4e27-981e-013810fe4cag img_v2_03aa9c30-dc36-4b00-b810-218fa690379g

Ste-RH commented 2 years ago

Whilst we could, we have a policy to not change the core ExoPlayer code - so we cannot implement your custom changes. You will need to take them to ExoPlayer themselves to get support added.

That aside, I was curious and so tried a version of ExoPlayer with your changes in. They do not work for me. Maybe because your code is looking for 12 channels, and the video you sent states it is 6.

Track:0, id=2, mimeType=audio/eac3-joc, channels=6, sample_rate=48000, language=und, supported=NO_UNSUPPORTED_TYPE

zyfeng413 commented 2 years ago

This mimeType information is not entirely correct because it is Dolby ATMOS, the mime information is only referable, and the actual output channel in our system is 12

zyfeng413 commented 2 years ago

We did a couple of comparison tests:

  1. ExoPlayer Demo in Car machine SOC: Regular video support but Dolby video Not Support
  2. ExoPlayer Demo with custom changes in Car machine SOC: Regular video and Dolby video all support
  3. AVPro v2.6.4 in Car machine SOC: Regular video and Dolby video all not support(no display and no sound)
  4. AVPro v1.9.4 in Car machine SOC: Regular video support but Dolby video Not Support(no sound)
  5. AVPro v2.6.4 in mobile phone: Regular video support but Dolby video Not Support(no sound)
  6. AVPro v1.9.4 in mobile phone: Regular video support but Dolby video Not Support(no sound)

Is there a way to fix this?

Chris-RH commented 2 years ago

As @Ste-RH mentioned, we could look into supporting Dolby video.

zyfeng413 commented 2 years ago

Thank you so much for your support. Look forward to good news on further support.

Ste-RH commented 2 years ago

As previously stated. We have a policy of not adding to the core ExoPlayer code that we distribute with AVPro Video.

It appears you need your custom code added to the ExoPlayer codebase in order for it to work.

If your changes are generic, then you would be best advised to raise it on the ExoPlayer issues board, or create a pull request and add them into the codebase for review yourself.

If your changes are custom, then I would think ExoPlayer will not entertain its inclusion, just as we will not at RenderHeads. There is a solutions though. You could build your own set of ExoPlayer libraries up with your changes and use those. The only change we make to the core codebase is changing the package name to 'com.google.android.exoplr2avp'. To find out what version of ExoPlayer your version of AVProVideo is using, you can filter your logcat to 'ExoPlayerLib'.

Ste-RH commented 2 years ago

Bit of housekeeping on this issue. Link to ExoPlayer issue: https://github.com/google/ExoPlayer/issues/10701

Ste-RH commented 2 years ago

The ExoPlayer dev team have made some changes to the library. This will naturally follow into AVPro Video when they next release a version of ExoPlayer and we integrate it into AVPro Video. Once @zyfeng413 confirms the changes fix the issue we will send over a new set of ExoPlayer libs for testing within AVPro Video to verify.

Ste-RH commented 1 year ago

Looks like there has been some code added to ExoPlayer v2.18.2 for this. We will update AVPro Video to use the latest release for ExoPlayer in our next release.

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Ste-RH commented 1 year ago

AVPro v2.7.0 was updated to use ExoPlayer v2.18.2. Do let us know if this addresses your issue @zyfeng413

Chris-RH commented 1 year ago

Any update @zyfeng413 ?

Chris-RH commented 1 year ago

I'm going to close this for now, as neither thread has had any contact from the OP for a while now. Please reopen if you still require assistance.