search

SRGSSR / srgmediaplayer-apple

An advanced media player library, simple and reliable
MIT License
158 stars 33 forks source link

tvOS support #61

Closed defagos closed 4 years ago

defagos commented 5 years ago

We should add tvOS support. Here is what has to do:

defagos commented 5 years ago

Work is making good progress on the feature/tvOS branch.

defagos commented 5 years ago

A quick look at the standard tvOS player view hiearchy:

tvOS player

defagos commented 5 years ago

Play / pause is obtained with tap gesture recognition:

<UITapGestureRecognizer: 0x600002598500; state = Possible; view = <_AVFocusContainerView 0x7f8854523020>; target= <(action=_handleMenuTapGestureOther:, target=<AVPlayerViewController 0x7f8856819a00>)>; must-fail-for = {
        <UITapGestureRecognizer: 0x600002599000; state = Possible; view = <_AVPlayerViewControllerContainerView 0x7f885640abd0>; target= <(action=_handleMenuTapGestureDismissal:, target=<AVPlayerViewController 0x7f8856819a00>)>>
    }>
)
defagos commented 5 years ago

The slider view is private and bears the progress bar itself as well as the time labels:

Slider

defagos commented 5 years ago

Trick play support (the ability to see a thumbnail of the location the user is scrubbing to) is important / mandatory for a good tvOS user experience.

This requires a EXT-X-I-FRAME-STREAM-INF playlist to be associated with the master playlist, see official HLS specifications.

The idea of using a secondary paused player to display the thumbnails is interesting, but I am not sure how we can officially access the inner playlist, at least without having a look at the m3u8 ourselves.

Remark: ExoPlayer support (long wanted) seems to be not so far away. See issue 474 and issue 6270. Trick mode support is documented in the DASH IOP, section 3.2.9.

There is probably even support from Akamai.

defagos commented 5 years ago

We should really consider if integrating with AVPlayerViewController is the way to go. The UX is standard and there is a lot we could use:

Have a look in particular at the documentation and the following WWDC videos:

https://developer.apple.com/videos/play/techtalks-apple-tv/6/ https://developer.apple.com/videos/play/wwdc2016/506/

A few challenges if we want to go this route are:

This is something I already tried in the past. I could not manage to ensure the controller state was correct at all times (especially during seeks), but still we might consider this needs further investigation.

defagos commented 5 years ago

Interstitials are the right way to ensure blocked regions cannot be peeked at (and are omitted in the slider range). They can even be updated during playback, which is nice since media player controller segments can as well.

defagos commented 5 years ago

There is a proper way to retrieve metadata for navigation markers when an async process is required. For common metadata, use AVMutableMetadataItem.

defagos commented 5 years ago

Using AVPlayerViewController works and could actually be implemented.

One thing to notice, though, is that one should apparently avoid having the player used by an AVPlayerViewController being attached to another AVPlayerLayer than the one managed by the AVPlayerViewController itself. Otherwise video playback will freeze in the simulator (audio & subtitles continue to advance) and, while everything seems to run fine on a device, it is probably best to ensure everything works well.

defagos commented 5 years ago

Here is how to setup a project (framework / tests / demo) so that it builds and runs on iOS, tvOS and watchOS, using a single target. These instructions assume a project is available with the following targets:

Project dependencies are assumed to be compatible with the targeted platforms (if this is not the case, apply these guidelines to these projects first, and point the Cartfile at compatible versions). Run the carthage command with the supported platforms to build these dependencies.

These instructions are inspired by an article from Dave DeLong, with a few additions to ensure better consistency among our projects:

  1. Clear the Base SDK for the project and all targets. Set Base SDK to iOS for app targets (this will be overridden automatically and correctly depending on the platform being built. Setting it for app targets has two benefits: The iOS icon is displayed next to the target if available, and code signing does not fail early when building device binaries).
  2. Clear all deployment target settings for all targets. Setup deployment target versions for all supported platforms at the project level.
  3. Set ch.srgssr.$(TARGET_NAME) as Product bundle identifier for the project and clear this setting in all targets.
  4. Set Product name as $(TARGET_NAME) at the project level and clear this setting in all targets (remark: For resource bundles, override with framework name to match our SRG SSR resource bundle name convention). Some characters are forbidden in product names (like hyphens). Use ${TARGET_NAME:c99extidentifier} in such cases.
  5. Clear Targeted device family setting for the project and all targets.
  6. Add .xcconfig files (see listings below), and set them for the specified targets:
    1. Common.xcconfig in a common location (usually project root). Not set for any target.
    2. Framework.xcconfig set for the framework target (and resource target if any).
    3. Tests.xcconfig set for the test target.
    4. Demo.xcconfig set for the demo target.
  7. If the framework has other framework dependencies (with Carthage), ensure that all Framework search paths reference $(CARTHAGE_PLATFORM) instead of the platform.
  8. For the test target:
    1. Under build phases, ensure that the test target only links against the project framework. Remove any Copy files phase which might copy frameworks into the destination Frameworks directory.
    2. Add $(PROJECT_DIR)/Carthage/Build/$(CARTHAGE_PLATFORM) to Framework search paths.
    3. Add $(FRAMEWORK_SEARCH_PATHS) to the Runpath search paths.
  9. For the demo target:
    1. Under build phases, ensure that the demo target only links against and embeds the project framework (remove all other embedded frameworks if any). Remove any Copy files phase which might copy frameworks into the destination Frameworks directory.
    2. For other framework dependencies, if any, use a Carthage copy-frameworks build phase. Use $(SRCROOT)/Carthage/Build/$(CARTHAGE_PLATFORM)/FrameworkName.framework as input for all framework dependencies.
    3. In the settings, set $(APP_ICONS_SOURCE) as Asset Catalog App Icon Set Name. Then use the standard asset catalog icon sets for setting icons. This variable is set by .xcconfig files.
    4. Use $(LAUNCH_SCREEN) for the Launch Screen name in Info.plist. This variable is set by .xcconfig files.
  10. (Optional, but cleaner) Remove duplicate frameworks in the project tree (identity the ones that can be removed by checking they aren’t used by any target).

The framework should now build for all platforms. Moreover, the demo and tests should build and run for all platforms as well. This might not be the case if some APIs are not available for all platforms. Moreover, some sources or resources might need to be specific to a platform. To work on products that can build on all platforms:

  1. Mark classes, enums, methods, etc. not meant for a platform with availability macros (e.g. __TVOS_PROHIBITED). In general, use these attributes instead of commenting out class interfaces with preprocessor macros so that the compiler can say that a class is not available for a specific platform when attempting to compile code referencing it. Note that __IOS_PROHIBITED also excludes tvOS. To mark tvOS-only items not available on iOS, we cannot therefore use this macro. For the same reason, API_UNAVAILABLE(ios) or API_AVAILABLE(tvos(9.0)) do not work alone. But combining them works, i.e. API_AVAILABLE(tvos(9.0)) API_UNAVAILABLE(ios) or API_AVAILABLE(tvos(9.0)) __IOS_PROHIBITED for an API available for tvOS only. For consistency, and in the case of an iOS + tvOS project, I recommend the following convention (which can be easily extended to a project supporting macOS or watchOS):
    • Mark classes, enums, etc. not available for tvOS with API_UNAVAILABLE(tvos).
    • Mark classes, enums, etc. not available for iOS with API_AVAILABLE(tvos(9.0)) API_UNAVAILABLE(ios), where the minimum supported tvOS version is mandatory.
  2. Use #if TARGET_OS_IOS / TV / etc. to conditionally exclude code from source files if only available for some platforms. This can also be used in a header file if needed. See TargetConditionals.h (also include this file if testing for platforms early in a header file, so that defines are properly set).
  3. Files meant for iOS / tvOS / watchOS only (source files, storyboards, xibs) can be automatically filtered by Xcode with ~ios / ~tvos / ~watchos suffixes respectively, so that they are only taken into account for their platform. This trick can also be used to have different implementations files for different platforms, instead of using too much preprocessor magic. See Dave DeLong's article for how this works (conventions are defined in the xcconfig files).
  4. For image catalogs:
    1. Only apply Universal if a resource is available on ALL platforms.
    2. If not available, check only for the specific platforms. For example, If a resource is available for iPhone, iPad and tvOS but not watchOS, uncheck Universal and check the three platforms.
    3. Hint: To copy an image between platforms, hold alt and drag the image to copy to the other platform.
  5. To retrieve storyboards or xibs, which are always separate for iOS / tvOS / watchOS, use a method like the one listed below.

Code signing

We use Automatic code signing, but usually Xcode creates a mess of various settings. To have a meaningful setup with no warnings:

  1. Click on each target and enable the automatic code signing checkbox on the dedicated page.
  2. Switch to the project settings view. cmd+click on the project and all its targets to display them side-by-side. Use Signing as filter to quickly get all relevant settings.
  3. Clean all settings to reset them to default values.
  4. Set Code Signing identity to Apple Development for the project. For resources and test bundles, set Code Signing Identity to Sign to Run Locally.
  5. Set a development team for demo targets which can be run on a device.

When properly setup, you should get the following results, with changes only made to three settings. The rest are default values:

signing

Common.xcconfig

Common settings are stored within a common xcconfig file, included by separate xcconfig files for the framework, demo and test target (which have room for additional specific parameters if needed).

// Configuration to have a single target built for all platforms
// See https://davedelong.com/blog/2018/11/15/building-a-crossplatform-framework/
SUPPORTED_PLATFORMS = iphoneos iphonesimulator watchos watchsimulator appletvos appletvsimulator
TARGETED_DEVICE_FAMILY = 1,2,3,4

CARTHAGE_PLATFORM[sdk=iphone*] = iOS
CARTHAGE_PLATFORM[sdk=appletv*] = tvOS
CARTHAGE_PLATFORM[sdk=watch*] = watchOS

// Setup to enable plaform suffixes to enable sources or resources for a specific platform only
// See https://davedelong.com/blog/2018/07/25/conditional-compilation-in-swift-part-2/
IOS_FILES = *~ios.*
TVOS_FILES = *~tvos.*
WATCHOS_FILES = *~watchos.*

EXCLUDED_SOURCE_FILE_NAMES = $(IOS_FILES) $(TVOS_FILES) $(WATCHOS_FILES)

INCLUDED_SOURCE_FILE_NAMES =
INCLUDED_SOURCE_FILE_NAMES[sdk=iphone*] = $(IOS_FILES)
INCLUDED_SOURCE_FILE_NAMES[sdk=appletv*] = $(TVOS_FILES)
INCLUDED_SOURCE_FILE_NAMES[sdk=watch*] = $(WATCHOS_FILES)

Framework.xcconfig

Assumes Common.xcconfig is one folder above.

#include "../Common.xcconfig"

Tests.xcconfig

Assumes Common.xcconfig is one folder above.

#include "../Common.xcconfig"

Demo.xcconfig

Assumes Common.xcconfig is one folder above.

#include "../Common.xcconfig"

LAUNCH_SCREEN[sdk=iphone*] = LaunchScreen~ios
LAUNCH_SCREEN[sdk=appletv*] =

APP_ICONS_SOURCE[sdk=iphone*] = AppIcon
APP_ICONS_SOURCE[sdk=appletv*] = App Icon & Top Shelf Image

REQUIRED_DEVICE_CAPABILITY[sdk=iphone*] = armv7
REQUIRED_DEVICE_CAPABILITY[sdk=appletv*] = arm64

Resources.m

NSString *ResourceNameForUIClass(Class cls)
{
    NSString *name = NSStringFromClass(cls);
#if TARGET_OS_TV
    return [name stringByAppendingString:@"~tvos"];
#else if TARGET_OS_WATCH
    return [name stringByAppendingString:@"~ios"];
#endif
}
defagos commented 4 years ago

The info panel has layout issues after updating the underlying information (e.g. the removing the chapter list, setting a smaller description, etc.). The reason is that the corresponding view controller, AVInfoPanelViewController, is laid out once for the AVPlayerViewController it depends on. Then the layout is reused, even if the information changes.

Here is how the panel is displayed internally (information obtained by disassembling AVKit for tvOS, located at /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVOS.platform/Library/Developer/CoreSimulator/Profiles/Runtimes/tvOS.simruntime/Contents/Resources/RuntimeRoot/System/Library/Frameworks/AVKit.framework/AVKit):

We should report the fact that the view controller never correctly updates to Apple. They can see this as a feature, I would rather say this is a bug, as the API does not prevent updating information, and the similar MPNowPlayingInfo can be updated at any time.

In the meantime, and for older tvOS versions, it is possible to remove the cached instance when a metadata update is detected, so that a new layout will be created for it:

Note that the panel does not behave smoothly when updates are made (e.g. chapter scroll position is lost, animations are interrupted), so AVPlayerItem tvOS metadata should only be changed when required.

defagos commented 4 years ago

I found an issue with DVR livestreams when played on tvOS. Sometimes AVPlayer only sees a duration of 0.

Our implementation was incorrectly returning on-demand as type in such cases. An improvement has been made (see commit 3fc78753d14e5f013bef7d055901826e79a5ac5c).