search

google / ExoPlayer

This project is deprecated and stale. The latest ExoPlayer code is available in https://github.com/androidx/media
https://developer.android.com/media/media3/exoplayer
Apache License 2.0
21.7k stars 6.02k forks source link

Difference in video quality compared to mediaplayer #5003

Closed loredana2314 closed 5 years ago

loredana2314 commented 5 years ago

Issue description

Hello We are using Exoplayer on our application. We have noticed that there is a difference on video quality between Exolplayer and Mediaplayer. The picture quality is better on Mediaplayer and the video plays smoothly (the colors seem to be different). On the other hand, picture on Exoplayer is not so clear and video freezes time after time. We are testing the same streams in both players

Version of ExoPlayer being used

The version of exoplayer is 2.7.1

Device(s) and version(s) of Android being used

We have tested on different devices, mainly in Android 4.4.2 STB

A full bug report captured from the device

tonihei commented 5 years ago

Could be related to #4559 which was fixed in version 2.9.0. Can you try updating to see if it still occurs? Besides that we would need example media to actually debug the problem. If you don't want to post it publicly, please send it to dev.exoplayer@gmail.com with "Issue #5003" in the subject.

loredana2314 commented 5 years ago

We tested with version 2.9.0 and the problem seems to continue. We tested on Amlogic and Rock cheap STB were the problem seems to continue. We also tested on Philips Android Tv and there is no problem. An example media is also sent to you as requested.

tonihei commented 5 years ago

I'm unfortunately getting 404s when trying to access the media you've sent. Can you check it's still valid?

loredana2314 commented 5 years ago

Sorry about that, we sent you another email with other media. Thank you for the support.

tonihei commented 5 years ago

The media plays fine for me, although I used a new device.

Can you give some more details of what's wrong with the ExoPlayer playback?

ojw28 commented 5 years ago

We've had some reports about AmLogic (and possibly RockChip) STBs in the past. For example https://github.com/google/ExoPlayer/issues/953. They are likely device specific issues that need reporting to the manufacturer, rather than to us.

loredana2314 commented 5 years ago

From our findings it looks like it is NOT a hardware/manufacturer issue. We also spoke to the device manufacture and they confirmed the same thing as below.

This are the results from our tests on the same device: Default player plays the stream very well. VLC player plays the the stream very well, same as default player. MX Player plays the stream very well, same as default player.

Video plays smoother and has more colors on above players compared to Exoplayer. On exoplayer we also notice some sporadic freezes from time to time.

Please let me know if you need further information. The hardware is running on an amlogic s812.

tonihei commented 5 years ago

There is also #926 and #678 which both sounds like the same problem. The conclusion on these issues was that it may be a device specific issue or may not occur with newer versions of ExoPlayer.

It's still a device-specific issue because it only happens on this device. However, given that the default player can play it fine, it seems we should be able to provide a workaround. Need to get hold of the device first though.

One thing that stands out in the logs is the repeated "AmlogicVideoDecoderAwesome: head is 001,do not add start code" in every renderer iteration which indicates that the decoder expects us not to send a start code.

It would be helpful to know the scope the problem. Can you check on all the devices where you've seen the problem which decoder they use? Look out for the "decoderInitialized" messages of the EventLogger. Can you also let us know the Build.DEVICE and Build.MODEL values of these devices? Thanks!

loredana2314 commented 5 years ago

From the logs "decoderInitialized" is OMX.amlogic.avc.decoder.awesome for video and OMX.google.aac.decoder for the audio. As for the Build.DEVICE: stvm8 and MODEL: Quad-Core Enjoy TV Box. Attached you will find some more logs from the device that we are interested in (the one that our customers have). You will also find system/etc/media_codecs.xml. We will also send an email with a new valid media example.

okycelt commented 5 years ago

Since our issue related to this topic was closed (#5022) and we also want this resolved, I'd like to share our observations here.

We're seeing basically the same symptoms as @loredanatosku. In MediaPlayer, the image quality is better (the image is sharper) and the playback is smoother. Whereas in ExoPlayer, the image is not that sharp and the video seems like it would have slightly lower frame rate than when playing in MediaPlayer.

We're seeing these issues only on some devices and some content though. Our DASH content is playing fine everywhere, but when we play the UDP multicasts from which our DASH content is prepared, on some devices, we see the problems described above. I've made a summary of the devices we tested on including the details @tonihei requested.

Problematic devices

Okay devices

If necessary, we can provide test content as well as bugreports from these devices.

Thanks for looking into this.

tonihei commented 5 years ago

@loredanatosku @okycelt Thanks for the input. We'll try to get hold of a device to reproduce ourselves. And then check if there is an easy workaround we could enable for those devices.

okycelt commented 5 years ago

@tonihei Have you made any progress regarding this? Thank you.

tonihei commented 5 years ago

No, sorry. Haven't got a device yet.

okycelt commented 5 years ago

@tonihei If you'd like, we would be willing to provide a test device for you. For free of course.

tonihei commented 5 years ago

Thanks for the offer, but that's probably too complicated. Would you reckon this device is one of the problematic ones?

okycelt commented 5 years ago

Would you reckon this device is one of the problematic ones?

Looking at the specs, I think it could be. The problematic box we use has the same specs.

okycelt commented 5 years ago

@tonihei Sorry to push, but do you have any estimate when approximately you're going to start working on this? Thanks again.

tonihei commented 5 years ago

Will investigate this week. Got one of the devices now.

okycelt commented 5 years ago

Great, thanks a lot.

okycelt commented 5 years ago

@tonihei Have you had a chance to take look at this? Thanks.

tonihei commented 5 years ago

I wasn't able to reproduce the problem with the streams provided by Loredana. And I couldn't test with your test stream so far as the link expired already - could you sent it again?

okycelt commented 5 years ago

I've just sent a link to dev.exoplayer@gmail.com

loredana2314 commented 5 years ago

Our stream is also OK now. It may have stopped for a moment but now it plays.

okycelt commented 5 years ago

@tonihei Have you been able to reproduce the problem with our stream?

tonihei commented 5 years ago

Haven't tried yet, sorry.

okycelt commented 5 years ago

@tonihei Could you keep us posted about the progress please? This si quite a blocking issue for us as a lot of our customers are waiting for this to be resolved to launch their service. I’m really sorry for pushing but we’re under quite a pressure.

ojw28 commented 5 years ago

We're already keeping you posted about progress.

From our findings it looks like it is NOT a hardware/manufacturer issue. We also spoke to the device manufacture and they confirmed the same thing as below.

If it's not a hardware/manufacturer issue, or at least something unusual or different about this particular device, then the same issue would reproduce on all other devices as well. Given this isn't the case, I'm finding it difficult to understand what logic you're using to come to this conclusion.

Other pieces of software not having the same issue isn't particularly strong evidence, since other pieces of software may well be using the underlying platform in different ways.

ojw28 commented 5 years ago

To put it another way, it seems reasonable to ask a device manufacturer why a piece of software that works fine on nearly all other devices doesn't work on their hardware, and to expect them to provide a response to this specific question. An acceptable response would be one that states what that piece of software is doing wrong in its interactions with the underlying platform (that other devices happen to handle in a more lenient way), or one that diagnoses there's a bug in their platform. A response that simply states some other piece of software works OK doesn't seem like an acceptable response.

okycelt commented 5 years ago

@ojw28 Device manufacturers are usually not helpful in this way as they don't understand ExoPlayer at all, or worse, don't want to understand it. That's why we're asking you for help, because you guys know what you're doing. If not a fix, we were hoping you could at least give us a hint what approximately could be the issue so we can forward it to the device manufacturer.

ojw28 commented 5 years ago

I don't think they really need to understand ExoPlayer. This is a bit hypothetical, since without knowing exactly what the issue is it's not possible to be sure, but I suspect the problem (or whatever is unusual/different about this platform compared to others) lies somewhere after the point where ExoPlayer feeds buffers to the video decoder. The path from there to video being displayed on the screen is almost entirely in the platform.

If it's an ExoPlayer issue, the hardware manufacturer should be able to say what they think is wrong with the buffers we're feeding into the video decoder. If they can't say what's wrong with the buffers being fed into the decoder, then that suggests that something is going wrong after that point. As I said, nearly all of the path after that point is in the platform. So if you have any relationship with the hardware manufacturer in question, I'd really suggest you push harder for a more specific answer.

We will also try and reproduce with the provided stream today or tomorrow. Thanks.

tonihei commented 5 years ago

Thanks for your patience! We were able to reproduce the problem now and also found a workaround :)

From our findings it looks like it is NOT a hardware/manufacturer issue.

It looks the issue is a bug in the "OMX.amlogic.avc.decoder.awesome" decoder on these devices, which clearly is a manufacturer issue. All players using this decoder fail in the same way (ExoPlayer, VLC with acceleration, Kodi.TV) whereas all players which are using something else work fine (MediaPlayer, VLC with software decoder, ExoPlayer with Google H264 software decoder).

Please note that MediaPlayer specifically can be implemented by the manufacturers as they like and this particular MediaPlayer does not use the standard platform decoders as far as we are able to tell.

See also here, here, and here for the same issue with the decoder elsewhere. We also tried making system changes as recommended in these issues, but none seem to solve the problem for us.

In ExoPlayer, the image is not that sharp and the video seems like it would have slightly lower frame rate.

It turns out that the decoder decides by itself to halve the output resolution of the video (which actually is 1920x1080). The logs show that twice, once from the decoder and once from ExoPlayer logging.

AmlogicVideoDecoderAwesome: omx output size changed from 1920x1088 to 960x540
...
EventLogger: videoSizeChanged [0.69, 0.00, window=0, period=0, 960, 540]

Not sure what exactly causes that to happen, but it's likely related to the fact that the video is interlaced and has a higher frame rate (which means that the number of lines per non-interlaced is only 540), but that's of course not a reason to do it wrong.

One other interesting thing that the logging shows is that the decoder pre-allocates memory for 1920x1088 before receiving any information about the actual video. This value seems to be hardcoded and independent of surface/screen/video sizes.

Workaround

It turns out that the problem only occurs if the decoder keeps using this pre-allocated memory. If we tell the decoder that our maximum expected input will be larger (in total number of pixels) than what it allocated memory for, it starts working fine. I guess that is because the wrong configuration gets invalided in such a case and the decoder sets itself up again with the actual media initialization data. So the suggested workaround would be to always claim that the maximum expected input size will be at least 1920x1089 (unless it's actually larger of course).

The downside of doing this is that the decoder will probably allocate that much memory even if the video is actually smaller or doesn't need the workaround. Given that it does that anyway upfront, it's supposedly not much of a difference when it comes to memory. But it may also add a small startup latency as a new memory reallocation is needed.

Based on the information you provided, it seems the target devices are devices with "OMX.amlogic.avc.decoder.awesome" and API level <= 25. That seems relatively broad. Do you have any suggestions to target that workaround more specifically?

Next steps.

  1. Can you verify this works for you as well? Please change these values here to be Math.max(codecMaxValues.width, 1920) and Math.max(codecMaxValues.height, 1089).
  2. Check all devices where such an issue was reported and if there is a way to provide a more targeted condition than the one I proposed above.
  3. We will also try to make Amlogic aware of the problem to see if they are willing to comment/change something.
okycelt commented 5 years ago

@tonihei, @ojw28 Thanks a lot for your support! It seems like the workaround works.

  1. Can you verify this works for you as well? Please change these values here to be Math.max(codecMaxValues.width, 1920) and Math.max(codecMaxValues.height, 1089).

On our side, changing those values does the trick as well. The video quality seems considerably better after the workaround.

  1. Check all devices where such an issue was reported and if there is a way to provide a more targeted condition than the one I proposed above.

I'm not sure if we are able to narrow the problematic devices down much. We have received several Amlogic devices from various STB vendors since we've reported the issue and all of them have fit this pattern so far. Also, we haven't seen the issue on devices from different chipset manufacturers.

loredana2314 commented 5 years ago

Thanks for your support! @tonihei Just to make sure about which values are to be changed since I don't have access to this : you are referring to maxWidth = Math.max(maxWidth, streamFormat.width) and maxHeight = Math.max(maxHeight, streamFormat.height) to be changed to maxWidth = Math.max(maxWidth, 1920) and maxHeight = Math.max(maxHeight, 1089) in MediaCodecVideoRenderer, right?

ojw28 commented 5 years ago

The link should have been this. I've updated the link in Toni's post above as well.

Svechnikov commented 5 years ago

I tried the workaround with setting the max values to 1920x1089 (which is already present in the last commit) and it didn't work on our amlogic s905 device (SDK 22) with 1920x1080 progressive video (in HLS). If I make method codecNeedsMaxVideoSizeResetWorkaround return false, it doesn't change anything.

However, if I make MediaFormat.KEY_HEIGHT equal to 1089, strange enough, the picture gets sharper. It looks like with height equal to any value except 1089 the picture gets interpolated (with bad algorithm), which results in poor quality.

For testing purposes I replaced the lines 1046 and 1047 (https://github.com/google/ExoPlayer/blob/r2.9.2/library/core/src/main/java/com/google/android/exoplayer2/video/MediaCodecVideoRenderer.java#L1046) with the following and picture became sharper: mediaFormat.setInteger(MediaFormat.KEY_WIDTH, 1920); mediaFormat.setInteger(MediaFormat.KEY_HEIGHT, 1089);

Though in logs I still see the line:

D/AmlogicVideoDecoderAwesome( 2825): omx output size changed from 1920*1088 to 960*540

tonihei commented 5 years ago

@Svechnikov Can you share a full bug report with all the logs in ExoPlayer's demo app? Preferably once with the working fix and once without. Would be good to see how they differ.

Svechnikov commented 5 years ago

@tonihei I modified my initial comment. And here's my full bug-report.

Issue description

On 1920x1080p video picture is not sharp, objects have pixelated edges. However, if I make MediaFormat.KEY_HEIGHT equal to 1089, strange enough, the picture gets sharper. It looks like with height equal to any value except 1089 the picture gets interpolated (with bad algorithm), which results in poor quality.

Reproduction steps

Launch the Apple master playlist advanced (TS) sample in ExoPlayer demo app on a tvbox connected to a full-HD screen and confirm, that the circle edges are not smooth.

After that replace this line https://github.com/google/ExoPlayer/blob/r2.9.2/library/core/src/main/java/com/google/android/exoplayer2/video/MediaCodecVideoRenderer.java#L1047 with the following:

mediaFormat.setInteger(MediaFormat.KEY_HEIGHT, 1089);

and confirm that the circle became smooth.

Link to test content

https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_ts/master.m3u8

Version of ExoPlayer being used

releaseVersion=2.9.4 releaseVersionCode=2009004

Device(s) and version(s) of Android being used

I observed this behavior only on Amlogic S905 (android sdk level 22) device. I also tested it on Amlogic s905x, there were no such problems.

A full bug report captured from the device

initial.bugreport.txt - bugreport without the test fix secondary.bugreport.txt - bugreport with the test fix Note: while launching the sample I had to choose stereo audio, otherwise the sample wouldn't start.

Here's a screenshot of the sample with pixelated edges. You can open if full-screen on a full-hd display and see that the circle isn't smooth. screen

Another interesting observation is that with the test fix video on screenshot (taken with screencap) becomes black (as if like MediaPlayer was used instead of ExoPlayer).

tonihei commented 5 years ago

The first bugreport unfortunately doesn't contain the start of the video where the problem happens. And the second one only has a weird size change to 160x90 pixels?

Can you verify again that directly setting the max values doesn't help too? mediaFormat.setInteger(MediaFormat.KEY_MAX_WIDTH, 1920); mediaFormat.setInteger(MediaFormat.KEY_MAX_HEIGHT, 1089);

Svechnikov commented 5 years ago

The first bugreport unfortunately doesn't contain the start of the video where the problem happens. And the second one only has a weird size change to 160x90 pixels?

I don't know where this weird size came from, but the circle in the test video was smooth.

Did tests one more time: testFix.log.txt - logs with MediaFormat.KEY_HEIGHT set to 1089 testInitial.log.txt - logs without any changes testMaxHeight.log.txt - logs with MediaFormat.KEY_MAX_HEIGHT and MediaFormat.KEY_MAX_WIDTH set to 1089 and 1920.

Can you verify again that directly setting the max values doesn't help too? mediaFormat.setInteger(MediaFormat.KEY_MAX_WIDTH, 1920); mediaFormat.setInteger(MediaFormat.KEY_MAX_HEIGHT, 1089);

Checked it another time, I confirm it doesn't help.

tonihei commented 5 years ago

Thanks for the reports. As you noted, it looks as if this device is ignoring the MAX_WIDTH and MAX_HEIGHT parameters completely. When changing the video size itself, the codec seems to switch to a special 4K code path which doesn't have the bug. Note how the line with "omx output size changed from 1920*1088 to ..." isn't even part of the output anymore.

Although changing the video size itself seems to work, it's may not be a good idea to change it in code because we would be giving the decoder a wrong configuration. Whereas setting a higher maximum size isn't wrong per se. As a whole bunch of other devices seem to work fine with the existing workaround (see above), I would leave it as it is in the hope that this is just an exception. Maybe also be worth reporting the issue to Amlogic again to see if they can provide a proper workaround.

Svechnikov commented 5 years ago

Although changing the video size itself seems to work, it's may not be a good idea to change it in code because we would be giving the decoder a wrong configuration. Whereas setting a higher maximum size isn't wrong per se. As a whole bunch of other devices seem to work fine with the existing workaround (see above), I would leave it as it is in the hope that this is just an exception. Maybe also be worth reporting the issue to Amlogic again to see if they can provide a proper workaround.

Yes, I understand that setting invalid size will necessary break playback on some other device and it would be a horrible decision to make such thing to ExoPlayer :)

ExoPlayer helps to get rid of vendor-specific bugs almost on every level, except hardware decoding.It would be great to have more deeper control over decoding process than existing MediaCodec has to offer.

tonihei commented 5 years ago

ExoPlayer helps to get rid of vendor-specific bugs almost on every level, except hardware decoding.It would be great to have more deeper control over decoding process than existing MediaCodec has to offer.

Well, the media codecs are part of the platform and we can't only work around issues or hope for vendor updates if possible. There are ExoPlayer extensions to use software decoders (e.g. ffmpeg for audio) if you prefer that.

tonihei commented 5 years ago

Closing this issue under the assumption that the workaround we committed helps.

giridharkannan commented 5 years ago

The given fix results in breaking the Amlogic box that I have. This results in a completely black screen which covers the whole viewing area (including any sibling views). I could hear only the audio.

If the new code is commented out, things work without any issue as expected.

It would be good if you could change the codecNeedsMaxVideoSizeResetWorkaround method from private static to protected so that I can override the behaviour.

I will also open up a new issue so that other users need not waste time on this issue.

Box Details: Device - G9X Manifacturer - ShiningWorth Android SDK - 23 Hardware - Amlogic

ojw28 commented 5 years ago

I think I'd be in favour of reverting the fix for now. It's unclear whether it's making things better, on balance (in particular if it's affecting playback of non-interlaced content).

tonihei commented 5 years ago

@mindfields5 We can't help you debug issues with custom modifications of the system parameters. It also doesn't sound like a particularly good fix if it changes the playback speed of the video when you rewind?

mindfields5 commented 5 years ago

We can't help you debug issues with custom modifications of the system parameters. It also doesn't sound like a particularly good fix if it changes the playback speed of the video when you rewind?

This is only problem we have discovered. You can specify these parameters on your STB using ADB.

mindfields5 commented 5 years ago

I uploaded a video where you can see problem. It is available at link - https://yadi.sk/i/yJ9SVWI5pENEpg

tonihei commented 5 years ago

@mindfields5 It's a customization we can't help you with because changing system parameters via adb is not something we can do from our library anyway.

tonihei commented 5 years ago

I'll close this issue again because the revert of the original fix has been pushed. If you find a way to target the max video size workaround to only the affected video streams, please let us know.