[PR Proposal] ASVideoNode performance improvement

[garrettmoon]

From @wokalski on October 31, 2016 9:58

I have recently came across the slow AVPlayer issue that has been discussed here and on the slack channel. I created a player using AVSampleBufferDisplayLayer which runs on a background queue and it works pretty good. I could submit a PR which changes AVPlayer in ASVideoNode to a substantially faster solution. Please note though that even though it's faster it is less predictable (runs on a background queue, built quickly) but stable (as in doesn't crash from what I've seen).

Let me know what you think

Copied from original issue: facebookarchive/AsyncDisplayKit#2513

[garrettmoon]

From @appleguy on October 31, 2016 22:36

@wokalski Hey there, thanks for investigating this and suggesting it!

We are very interested in higher performance implementations for ASVideoNode. There are some hacky techniques to do this with AVPlayer, but not all of those techniques can be upstreamed, leaving the public version suboptimal for real apps.

What iOS version does your new implementation work on? What are the less predictable elements — does framerate of playback vary, or any issues with audio support?

I would suggest publishing either a new mode of operation for ASVideoNode that we may enable to be the default later (perhaps even for the 2.0 release), or creating an ASVideoNodeBeta that we could cut over to later (we've done this before with ASRangeController being rewritten). This would allow us to accept the changes into master as quickly as possible, and get other developers testing and moving it forward, without exposing all users to the initial risk.

[garrettmoon]

From @wokalski on November 1, 2016 13:10

thanks for the detailed reply.

I built a prototype app (keep this in mind) for a client. It had a list of videos looped simultaneously without audio. 720p 3-4 played at the same time (playback activated when a cell enters the screen). I don't know if it matches your AVPlayer experience but play and pause blocked main thread for long periods.

I used AVSampleBufferDisplayLayer (iOS 8+). It uses a background queue to request next frames (so to speak). This states the solution and yields numerous doubts about predictability.

1. Does framerate vary? Since everything happens on a background queue, the predictability of the solution depends on the resources available. For instance with 3-4 1080p videos played simultaneously on iPhone 6 the issue was noticeable (I don't have numbers). Dropping frames were also visible when I used the device as a hotspot and recorded the screen. However in general it worked perfectly and I believe the issues above are understandable and should be handled on application level. (don't play 4 videos simultaneously if you don't want dropped frames)
2. Audio support I didn't use audio but it shouldn't be a problem. We specify the timing properties of a layer. Somebody suggested using CMAudioClock to time it correctly but I didn't need it (no audio)

Also please note that my solution is bare bones. It might have more catches but it boils down to acceptable drawbacks. In my use case 60fps of the interface was more important than few moments of worse framerate of the player. With AVPlayer it freezes the whole UI so I think of it as an improvement.

When it comes to the publishing it, ASVideoNodeBeta sounds like a great solution. I believe it can be further improved (and made more predictable) but it requires more effort than a few hours I can spend on it now.

If you have any doubts (I wouldn't be surprised) I can reimplement the video example as a proof of concept.