search

FDH2 / UxPlay

AirPlay Unix mirroring server
GNU General Public License v3.0
1.35k stars 72 forks source link

how to support airplay video #101

Closed ruoying86 closed 2 years ago

ruoying86 commented 2 years ago

UxPlay Highlights: Support for both AirPlay Mirror and AirPlay Audio-only (Apple Lossless ALAC) streaming protocols from current iOS/iPadOS 15.4 clients.

this mean not support airplay video?

why not support airplay video

now i use uxplay can do airplay mirror , but youku push can only push sound

monomycelium commented 2 years ago

AirPlay Video is supported on UxPlay only to mirror the screen of your iPhone, iPad, iPod touch, or Mac to your device. However, content from some streaming platforms like Apple TV+ and Netflix cannot be directly streamed because of DRM. If your content does not have DRM, see #44.

ruoying86 commented 2 years ago

i use youku app push no video，log show ：ct=2 spf=352 usingScreen=0 isMedia=0 audioFormat=0x40000

what is wrong with usingScreen=0 or isMedia=0 ？

i check #44，youtube_app_airplay_video.txt log，isMedia=1

log airplay_youku_no_video.txt

fduncanh commented 2 years ago

ct=2 spf=352 usingScreen=0 isMedia=0 audioFormat=0x40000

This is the Apple Airplay sound-only protocol. (ALAC loss-free audio). It does not send any video, even to a "real" Apple TV.

Apple music uses ALAC. You need to use Airplay Mirror to send video from your iPad/iPhone screen. There are two different icons on the iPad/iPhone. One (two overlapping rectangles representing video screens) for Airplay Mirror, one for AirPlay Audio-Only ( a triangle with circles of radio waves around its peak)

ruoying86 commented 2 years ago

@fduncanh iphone 5s version ios10.3.3 ,the same step with redmi tv，can stream video ，but worked with uxplay ，only stream audio iphone 7p version ios15.1 ，the same step with uxplay，log is ct=2 spf=352 usingScreen=0 isMedia=1 audioFormat=0x40000。but raop_ntp receive timeout

ruoying86 commented 2 years ago

refer to https://github.com/antimof/UxPlay/issues/14 https://github.com/FD-/RPiPlay/pull/146#issuecomment-675134133 https://gstreamer.freedesktop.org/documentation/installing/on-linux.html?gi-language=c

sudo apt-get install gstreamer1.0-doc E: Package 'gstreamer1.0-doc' has no installation candidate

sudo apt-get install libgstreamer-plugins-bad1.0-dev

Err:30 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libgdcm-dev amd64 3.0.10-1build2
Connection failed [IP: 91.189.91.38 80] Err:31 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libgl2ps1.4 amd64 1.4.2+dfsg1-2
Connection failed [IP: 91.189.91.38 80] Err:32 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libglew2.2 amd64 2.2.0-4
Connection failed [IP: 91.189.91.38 80] Err:33 http://cn.archive.ubuntu.com/ubuntu jammy/main amd64 libgphoto2-dev amd64 2.5.27-1build2
Connection failed [IP: 91.189.91.38 80] Err:34 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libhwloc15 amd64 2.7.0-2
Connection failed [IP: 91.189.91.38 80] Err:35 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libilmbase-dev amd64 2.5.7-2
Connection failed [IP: 91.189.91.38 80] Err:36 http://cn.archive.ubuntu.com/ubuntu jammy/main amd64 libjpeg-turbo8-dev amd64 2.1.2-0ubuntu1
Connection failed [IP: 91.189.91.38 80] Err:37 http://cn.archive.ubuntu.com/ubuntu jammy/main amd64 libjpeg8-dev amd64 8c-2ubuntu10
Connection failed [IP: 91.189.91.38 80] Err:38 http://cn.archive.ubuntu.com/ubuntu jammy/main amd64 libjpeg-dev amd64 8c-2ubuntu10
Connection failed [IP: 91.189.91.38 80] Err:39 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 liblept5 amd64 1.82.0-3build1
Connection failed [IP: 91.189.91.38 80] Err:40 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libtbbmalloc2 amd64 2021.5.0-7ubuntu2
Connection failed [IP: 91.189.91.38 80] Err:41 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libtbb2 amd64 2020.3-1ubuntu3
Connection failed [IP: 91.189.91.38 80] Err:42 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-core4.5d amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:43 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-flann4.5d amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:44 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-imgproc4.5d amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:45 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-features2d4.5d amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:46 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-calib3d4.5d amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:47 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libtbb12 amd64 2021.5.0-7ubuntu2
Connection failed [IP: 91.189.91.38 80] Err:48 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libtbb-dev amd64 2021.5.0-7ubuntu2
Connection failed [IP: 91.189.91.38 80] Err:49 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-core-dev amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:50 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-flann-dev amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:51 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-imgcodecs4.5d amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:52 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-highgui4.5d amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:53 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-imgproc-dev amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:54 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-imgcodecs-dev amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:55 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-videoio4.5d amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:56 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-videoio-dev amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:57 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopenexr-dev amd64 2.5.7-1
Connection failed [IP: 91.189.91.38 80] Err:72 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-dnn-dev amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:73 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-objdetect-dev amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:74 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-photo4.5d amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:75 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-photo-dev amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:76 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-shape4.5d amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:77 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-video-dev amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:78 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-shape-dev amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:79 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-stitching4.5d amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:80 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-stitching-dev amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:81 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-superres4.5d amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:82 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-superres-dev amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:83 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-videostab4.5d amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:84 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-videostab-dev amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:85 http://cn.archive.ubuntu.com/ubuntu jammy/main amd64 libxnvctrl0 amd64 510.47.03-0ubuntu1
Connection failed [IP: 91.189.91.38 80] Err:86 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libhwloc-plugins amd64 2.7.0-2
Connection failed [IP: 91.189.91.38 80] Err:87 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libpmix2 amd64 4.1.2-2ubuntu1
Connection failed [IP: 91.189.91.38 80] Err:88 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libucx0 amd64 1.12.1~rc2-1
Connection failed [IP: 91.189.91.38 80] Err:89 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopenmpi3 amd64 4.1.2-2ubuntu1
Connection failed [IP: 91.189.91.38 80] Err:90 http://cn.archive.ubuntu.com/ubuntu jammy/main amd64 libtk8.6 amd64 8.6.12-1build1
Connection failed [IP: 91.189.91.38 80] Err:91 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libvtk9.1 amd64 9.1.0+really9.1.0+dfsg2-3build1
Connection failed [IP: 91.189.91.38 80] Err:92 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-viz4.5d amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:93 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-viz-dev amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:94 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-contrib-dev amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:95 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv-dev amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:96 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv4.5d-jni amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:97 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libopencv4.5-java amd64 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80] Err:98 http://cn.archive.ubuntu.com/ubuntu jammy/main amd64 libpng-tools amd64 1.6.37-3build5
Connection failed [IP: 91.189.91.38 80] Err:99 http://cn.archive.ubuntu.com/ubuntu jammy/main amd64 libraw1394-tools amd64 2.1.2-2build2
Connection failed [IP: 91.189.91.38 80] Ign:100 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 opencv-data all 4.5.4+dfsg-9ubuntu4
Err:101 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 gir1.2-gst-plugins-bad-1.0 amd64 1.20.1-1ubuntu2
Connection failed [IP: 91.189.91.38 80] Err:102 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libgstreamer-opencv1.0-0 amd64 1.20.1-1ubuntu2
Connection failed [IP: 91.189.91.38 80] Err:103 http://cn.archive.ubuntu.com/ubuntu jammy/main amd64 libgstreamer-plugins-good1.0-dev amd64 1.20.1-1ubuntu1
Connection failed [IP: 91.189.91.38 80] Err:104 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 libgstreamer-plugins-bad1.0-dev amd64 1.20.1-1ubuntu2
Connection failed [IP: 91.189.91.38 80] Err:105 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 proj-bin amd64 8.2.1-1
Connection failed [IP: 91.189.91.38 80] Err:100 http://cn.archive.ubuntu.com/ubuntu jammy/universe amd64 opencv-data all 4.5.4+dfsg-9ubuntu4
Connection failed [IP: 91.189.91.38 80]

raop_ntp receive timeout not fix

ruoying86 commented 2 years ago

@fduncanh UxPlay is not a full emulation of Apple TV, In particular it doesn't let the client run a http server on the server, which is probably what the YouTube app wants to do. the note in https://github.com/antimof/UxPlay/issues/14，is mean Push video from YouTube app is not working fine with uxplay？the issue is only audio？

fduncanh commented 2 years ago

@ruoying86

Unfortunately, UxPlay cannot play video from YouTube.app directly, it can only play the sound in AirPlay-Audio-only ALAC mode, like Apple Music. Same is for AppleTv.app.

Since your redmi tv works, some hacker could investigate how it does it, and what the protocol is. Probably has extra encryption.

This would be a major task for a talented hacker, unless someone already did it.

UxPlay is using "LegacyPairing" mode (bit 27 of the "features" string it reports to the client is set). Maybe this tells the client not to send DRM video? I have no idea.

Here is how the protocol used by UxPlay was originally discovered

https://www.itread01.com/hkeyxif.html (in Chinese original)

https://github.com/FDH2/UxPlay/wiki/AirPlay2 (same thing, translated into English)

https://github.com/SteeBono/airplayreceiver/wiki/AirPlay2-Protocol https://emanuelecozzi.net/docs/airplay2 http://github.com/FD-/RPiPlay#airplay-protocol-versions

fduncanh commented 2 years ago

I think everything EXCEPT appleTV.app can be viewed by UxPlay in AirPlay-Mirror mode. (Use the iOS mirror icon for this (two rectangles that overlap), NOT the in-app AirPlay icon)

https://support.apple.com/en-us/HT204289

appleTV.app is "special": It justs shows a black screen within a frame in mirror mode, and you may even have to shut down and restart the iPad/iPhone after trying to play appleTV.app on UxPlay, before UxPlay will work again! Some bad DRM seems to get started on the iOS device that will not go away till you restart!

fduncanh commented 2 years ago

Re. installing gstreamer on ubuntu.:

fduncanh commented 2 years ago

This is a feature request that can't be implemented at this time, because the protocol for it is unknown.

Also it would not be a good idea to incorporate an http server into uxplay, if that is what would be needed.

ruoying86 commented 2 years ago

airplay_youku_video This is the interactive process of pushing youku video with airplay The handshake protocol seems to be similar to audio video interactive protocol uses the http protocol，need to support http server

Why incorporate an http server into uxplay is not be a good idea？ the wireshark file seems to need an http server

fduncanh commented 2 years ago

So long as there is no unknown encryption/decryption going on after pair verify I suppose it could be done.

UxPlay uses the "legacyPairing" protocol in which all client server communication remains not encrypted after pair verify.

The RpiPlay author noted that

From captures of the traffic between an iOS device running iOS 12.2 and an AppleTV running tvOS 12.2.1, one can see that the communication on the main mirroring HTTP connection is encrypted after the initial handshake. This could theoretically be part of the new AirPlay 2 protocol. The AppleTV running tvOS 12.2.1 identifies as AirTunes/380.20.1. When connecting from the same iOS device to an AppleTV 3rd generation (reporting as AirTunes/220.68), the communication is still visible in plain. From the log messages that the iOS device produces when connected to an AppleTV 3rd generation, it becomes apparent that the iOS device is treating this plain protocol as the legacy protocol (as originally introduced with iOS 9). Further research showed that at the moment, all available third-party AirPlay mirroring receivers (servers) are using this legacy protocol, including the open source implementation of dsafa22, which is the base for RPiPlay. Given Apple considers this a legacy protocol, it can be expected to be removed entirely in the future. This means that all third-party AirPlay receivers will have to be updated to the new (fully encrypted) protocol at some point.

More specifically, the encryption starts after the pair-verify handshake completed, so the fp-setup handshake is already happening encrypted. Judging from the encryption scheme for AirPlay video (aka HLS Relay), likely two AES GCM 128 ciphers are used on the socket communication (one for sending, one for receiving). However, I have no idea how the keys are derived from the handshake data.

The LegacyPairing is enabled by bit 27 of the "features" code in the plist initially sent by UxPlay to the client. Let's hope that it is still supported when iOS 16.0 is released later this year.

If the client-server communications you see with WireShark are unencrypted, they are probably in plist format. An example of a plist header is

RTSP/1.0 200 OK 
CSeq: 0 
Server: AirTunes/220.68 
Content-Type: application/x-apple-binary-plist 
Content-Length: 1063

If you can read the wireshark messages, maybe they are not encrypted. Why dont you post them here? (as a text file) I will tell you what I think they are.

You are welcome to fork Uxplay and work on it! Right now, Uxplay just (1) sets up cryptography pairing with the iOS device so it can decrypt the video and audio streams; (2) sets up an ntp server to keep the audio and video synchronized so video matches audio; (3) streams the separate decrypted video and audio streams with synchronized timestamps into separate GStreamer pipelines.

Adding a http:server would complicate things, and maybe be a security issue. I personally would not have time to work on this, my contribution to UxPlay is now basically completed after adding support for firewalls, ALAC audio-only and now Raspberry PI with Video4Linux2. (RpiPlay only does OpenMAX, which is no longer being supported on future operating systems for the Pi, as is it is 32-bit only.)

ruoying86 commented 2 years ago

the wireshark file： airplay_youku_video.txt the client-server communications after pair verify are unencrypted。the server version is AirTunes/220.68 as expected The data responded by the server is in plist format Currently it can be confirmed that IOS 15.3.1/15.4.1 with AirTunes/220.68 server can work

fduncanh commented 2 years ago

Since things are not encrypted, it looks possible to perhaps understand the protocol that is used for AirPlay video streaming to smart tv's.

There is a lot of recent work on AirPlay2 audio for streaming synchronised sound to smart speakers etc. (shairport-sync) but I dont know any similar projects for video. You would be taking on a big project if you try!. Good luck if you do, it would eat up a lot of your free time! I'm sorry I cant help.

thiccaxe commented 2 years ago

FWIW I am doing some research on how the YouTube app airplays. (On AppleTV3,1 - probably same for 3,2)

First thing I noticed is that 3 connections are negotiated /pair-verify etc.

Second thing is that the first connection makes a request GET /server-info which returns xml/plist (usually it is /info with bplist)

Third thing is that on the second connection, client makes the request POST /fp-setup2. I have not seen this url documented anywhere. It is in similar format to the second /fp-setup with a similar length of 164 bytes (have not counted)

I have yet to discover exactly what allows these alternate steps. However, one thing I found out is that with the current UxPlay dnssd "features" integer, YouTube will send only audio. However, change the dnssd features number to the one found in Apple TV 3 and YouTube will attempt to (and fail) to negotiate video AND audio with UxPlay.

Also, the Airplay client version number has been bumped in the iOS 16 beta 1 however they have not dropped support for Apple TV 3 (yet)

fduncanh commented 2 years ago

Re: iOS16. We live in fear every major iOS version... ! Its not AppleTV3 that matters, its "LegacyPairing" (bit 27)

There are lots of smart TV's that do AirPlay2 (except for AppleTV-app DRM) There is as example in https://github.com/SteeBono/airplayreceiver/wiki/AirPlay2-Protocol (which is the wiki of a project that seems essentially to be a direct transcription of (probably) dsafa22's code into c#). Also @ruoying86 is reporting about a China smart-tv "redmi-tv".

If the smart-tv's use LegacyPairing it wont go away soon, I guess. It would be useful to have reports.

Edit: AppleTV3 is already unsupported as far as activation is concerned. I picked up one for USD20 on ebay six months ago, and erasing then reactivating it bricked it. It no longer can connect directly to the activation server, it seems. Luckily one can still activate it using Apple Configurator 2.app running on a Mac, with the Apple TV3 connected via usb (this is the app for changing MDM settings on corporate iOS devices), and there is some windows app that can also do this.

EDIT2

actually, still supported: I just switched on my AppleTV3,2 for the first time in months, and got a software update to "7.9 (8163)"

fduncanh commented 2 years ago

@thiccaxe can you document the features integers (both Apple TV3 and Uxplay) you mention. Both https://github.com/SteeBono/airplayreceiver/wiki/AirPlay2-Protocol and https://emanuelecozzi.net/docs/airplay2 have tables of the bit values (neither are complete, but they are consistent, use both) and translate the features integers. into both hex and binary. I am guessing it is bit 49 that is needed for airplay2 video.

UxPlay seems to report 130367356919 = 0x1E5A7FFFF7= 1 1110 0101 1010 0111 1111 1111 1111 1111 0111

It may be "lying" about supporting some things. I see I should probably switch off bit 15, since artwork (jpeg) is just discarded when it is received by UxPlay in AirPlay-Audio-only mode

If I am interpreting this correctly 

ThirdPartyTV (26 || 51) && (0 || 49)

bit | value
63 0
62 0
61 0
60 0

59 0
58 0
57 0
56 0 (no) SupportsWoL 55 || 56

55 0 (no) SupportsWoL 55 || 56
54 0 (no) SupportsAPSync
53 0
52 0 (no) SupportsSetPeersExtendedMessage

51 0 (no) SupportsUnifiedPairSetupAndMFi
50 0 (no) MetadataFeatures_3 "Send NowPlaying info via bplist" 
49 0 (no) SupportsAirPlayVideoV2
48 0 (no) SupportsCoreUtilsPairingAndEncryption     38 || 46 || 43 || 48

47 0
46 0 (no) SupportsHKPairingAndAccessControl
45 0
44 0

43 0 (no) SupportsSystemPairing
42 0 (no) SupportsScreenMultiCodec
41 0 (no) SupportsPTP "Bit needed for device to show as supporting multi-room audio"
40 0 (no) SupportsBufferedAudio "Bit needed for device to show as supporting multi-room audio"
39 0
38 0 (no) SupportsUnifiedMediaControl
37 0
36 1

35 1 (yes)  SupportsTLS_PSK
34 1 (yes) SupportsAirPlayFromCloud
33 1  (yes) SupportsAirPlayVideoPlayQueue
32 0  (no)  IsCarPlay/Supports Volume "Don’t read key from pk record it is known"

31 0 
30 1 (yes) HasUnifiedAdvertiserInfo
29 0
28 1

27 1 (yes) SupportsLegacyPairing |   |  
26 0 (no) InfoAuthentication_8 = 26 || 51 " MFi authentication"
25 1 
24 0

23 0  (no) authentication type 1 RSA authentication
22 1
21 1 (yes) AudioFormats_3 "support for audio format 4"
20 1 (yes) AudioFormats_2 "support for audio format 3. This bit must be set for AirPlay 2 connection to work"

19 1 (yes) AudioFormats_1 "support for audio format 2. This bit must be set for AirPlay 2 connection to work"
18 1 (yes) AudioFormats_0  "support for audio format 1"
17 1 (yes) MetadataFeatures_2 "Send NowPlaying info via DAAP"
16 1 (yes) MetadataFeatures_1 "Send track progress status to receiver"

15 1 (yes) MetadataFeatures_0 "Send artwork image to receiver"
14 1 (yes) Authentication_4   "FairPlay authentication"
13 1 (yes) PhotoCaching "photo preloading supported"
12 1 (yes) FPSAPv2pt5_AES_GCM "FairPlay secure auth supported"

11 1 (yes) AudioRedundant
10 1
9   1 (yes) SupportsAirPlayAudio "audio supported"
8   1 (yes) ScreenRotate "screen rotation supported"

7   1 (yes) SupportsAirPlayScreen
6   1
5   1 (yes) SupportsAirPlaySlideshow
4   1 (yes) VideoHTTPLiveStreams "http live streaming supported"

3   0  (no) volume control for videos
2   1 (yes) video protected with FairPlay DRM
1   1 (yes) SupportsAirPlayPhoto
0   1 (yes) SupportsAirPlayVideoV1

steebono's samsung tv wireshark trace shows 0x7F8AD0, 0x38BCCB46 =0111 1111 1000 1010 1011 0000 , 0011 1000 1011 1100 1100 1011 0100 0110

bit 49 = 1   airplayvideo v2
 bit 0 = 0    airplayvideo v1
bit 26 = 0
bit 51 = 1
thiccaxe commented 2 years ago

Edit: AppleTV3 is already unsupported as far as activation is concerned. I picked up one for USD20 on ebay six months ago, and erasing then reactivating it bricked it. It no longer can connect directly to the activation server, it seems. Luckily one can still activate it using Apple Configurator 2.app running on a Mac, with the Apple TV3 connected via usb (this is the app for changing MDM settings on corporate iOS devices), and there is some windows app that can also do this.

I purchased an Apple TV3,1 in January this year and was able to set it up without issue. Last time I checked it was able to use apple services. Perhaps it is due to something else such as region? not sure

Re: iOS16. We live in fear every major iOS version... !

in the ios 16 beta one thing I noticed was that when a connection to UxPlay fails (as mentioned in prev. comment) ios 16 will 'drop' that apple tv from the airplay list. Unsure as to exactly what allows it to reappear. This could be a beta bug or a cause of some of the airplay changes.

fduncanh commented 2 years ago

@thiccaxe so what is the features integer on your apple TV3. I guess I cant see it on mine without wireshark?

thiccaxe commented 2 years ago

I use this website which provides a nice interface: https://openairplay.github.io/airplay-spec/features.html and seems to be the same data.

While UxPlay does indeed report 130367356919 = 0x1E5A7FFFF7 , that is in the HTTP connection with the plist. The clue is that it is a different integer in dnssdint.h: 0x5A7FFEE6. In this case it is actually good,

UxPlay plist features integer is 130367356919 = 0x1E 5A7FFFF7 which is same as the one from AppleTV which is 61647880183 = 0xE 5A7FFFF7 but with bit 36 enabled.

With the setup 0xE << 32 | 0x5A7FFFF7 in raop_handlers.h and AIRPLAY_FEATURES:

for other apps like Vimeo, I have not tested as much but the behavior varies

thiccaxe commented 2 years ago

I guess I cant see it on mine without wireshark? you can use https://pyatv.dev use atvremote. Or you could use something like Avahi Zerconf Browser to see what the mDNS record of the apple tv is.

fduncanh commented 2 years ago

Re apple3,2 activation. Probably they fixed the activation server. At end of December 2021 when I got mine, when the reset bricked it, I found a number of discussions (angry posts!) about this on the internet, which led me to the apple configurator fix.

fduncanh commented 2 years ago

atvremote scan worked for me

but I couldnt get it to show "features" of the Apple TV3

This info seems useful about AirPlay2 https://pyatv.dev/documentation/supported_features/

fduncanh commented 2 years ago

In particular. they (pyatv) say:

Things change. Constantly. Here are a few things worth knowing about the protocols:

fduncanh commented 2 years ago

@thiccaxe so you are basically reporting that 0x5A7FFFF7 in plist should change to 0x5A7FEE6?

the change in dnssdint.h occurred here in RPiPlay, without the corresponding change being made in the plist in raop_handlers.h

It seems that features is set in 3 places, in _airplay.tcp, _raop.tcp, and in the plist in raop_handlers.h

fduncanh commented 2 years ago

@thiccaxe I fixed the plist to in raop_handlers.h to take values from dnssdint.h, so its now 0x5A7FEE6

https://github.com/FDH2/UxPlay/commit/63092afcd6daf8cc0e298683b3f9a38d4f1695e0

but I dont see that youtube is behaving differently. Its either in airplay-mirror mode, or airplay-audio without video. Am I missing something?

thiccaxe commented 2 years ago

Sorry, I can see now that my previous comment was confusing.

The idea is that iOS will look at the mDNS features record of the Apple TV first, to figure out if it supports (for example, in the YouTube case, the Video bit). Based on that it will continue to either screen mirror, or use the http link stream.

the features value sent in the plist does not affect this decision and is (was) fine. I have not tested with the feature value from dnssdint.h but for UxPlay's use of video (screen mirroring) everything should work fine.

fduncanh commented 2 years ago

@thiccaxe

In any case, the inconsistent plist seems to be an error (with no consequences) introduced in RPiPlay when the features were changed (..FF7 -> ..EE6) away from those of the AppleTV3 in dnsssdint.h but not in raop_handlers.h that originally had a binary copy of the plist taken from the AppleTV, which was then transcribed into plist from.

But what exactly did you test (what was changed and where?) I guess I didn't understand

thiccaxe commented 2 years ago

I changed EE6 to FF7 in the in the mDNS features, and apps like YouTube will NOT work, even while using screen mirror (two rectangles).

fduncanh commented 2 years ago

Your tests were with UxPlay or with the real AppleTV3? I'm still confused!

This is what I have done in dnssdint.h: (the plist in raophandlers.h now takes features = AIRPLAY_FEATURES_2 << 32 | AIRPLAY_FEATURES_1 (after converting strings to uint64_t)

the change 5A7FFFF7 to 5A7FFFE6 seems to have been make for the reasons you find. (the later change 5A7FFFE6 to 5A7FFEE6 is to do with rotating the ipad which changes between portrait and landscape

But I am not sure what to look for on youtube.

#define RAOP_TXTVERS "1"
#define RAOP_CH "2"             /* Audio channels: 2 */
#define RAOP_CN "0,1,2,3"       /* Audio codec: PCM, ALAC, AAC, AAC ELD */
#define RAOP_ET "0,3,5"         /* Encryption type: None, FairPlay, FairPlay SAPv2.5 */
#define RAOP_VV "2"
#define FEATURES_1 "0x5A7FFEE6" /* first 32 bits of features */
#define FEATURES_2 "0x0"        /* second 32 bits of features */
#define RAOP_FT FEATURES_1 "," FEATURES_2
#define RAOP_RHD "5.6.0.0"
#define RAOP_SF "0x4"
#define RAOP_SV "false"
#define RAOP_DA "true"
#define RAOP_SR "44100"         /* Sample rate: 44100 */
#define RAOP_SS "16"            /* Sample size: 16 */
#define RAOP_VS GLOBAL_VERSION  /* defined in global.h */
#define RAOP_TP "UDP"           /* Transport protocol. Possible values: UDP or TCP or TCP,UDP */
#define RAOP_MD "0,1,2"         /* Metadata: text, artwork, progress */
#define RAOP_VN "65537"
#define RAOP_PK "b07727d6f6cd6e08b58ede525ec3cdeaa252ad9f683feb212ef8a205246554e7"

/* use same features for RAOP and AIRPLAY: is this correct? */
#define AIRPLAY_FEATURES_1 FEATURES_1
#define AIRPLAY_FEATURES_2 FEATURES_2
#define AIRPLAY_FEATURES  AIRPLAY_FEATURES_1 "," AIRPLAY_FEATURES_2 
#define AIRPLAY_SRCVERS GLOBAL_VERSION /*defined in global.h */
#define AIRPLAY_FLAGS "0x4"
#define AIRPLAY_VV "2"
#define AIRPLAY_PK "b07727d6f6cd6e08b58ede525ec3cdeaa252ad9f683feb212ef8a205246554e7"
#define AIRPLAY_PI "2e388006-13ba-4041-9a67-25dd4a43d536"
fduncanh commented 2 years ago

@thiccaxe

Yes, I verified your report that the important thing for youtube.app to play in uxplay mirror mode is that the features code in _airplay._tcp has bit 0 switched off. This is what is advertised in mDNS_SD It doesn't seem to care about _raop._tcp or the plist list, it seems.

It seems correct to make all three places where features are reported consistent.

thiccaxe commented 2 years ago

I have done some wireshark debugging, and it seems that a DMAP (port 3689) server is required for apps like YouTube to function. (this makes sense, as DMAP has API for scrubbing, pause/play, etc., see https://pyatv.dev/documentation/protocols/#digital-media-access-protocol-dmap) I think that this is now well out of scope for UxPlay, in addition to the required http client and will work on a fork.

Additionally, there is no need to clutter here aswell.

Best of luck

fduncanh commented 2 years ago

@thiccaxe A fork is a good idea.

I believe that the UxPlay codebase has advanced a bit from RPiPlay, and even the 2019 dsafa22 sources, with some fixes and better understanding of the RAOP protocol, but it is basically finished, for what it is. Its use with an iPad with Apple pencil for online teaching with Zoom on linux was an important initial motivation for this fork. Adding support for video streaming with a http client and DMAP is a new direction, and should be a new forked project .

I se the POST/scrub requests in @ruoying86 's post above. Yes that seems to be DMAP.

Happy forking! The pyatv.dev site seems very useful! Maybe it has details of non-Legacy authentication protocols?

thiccaxe commented 1 year ago

On YouTube version 17.46.4 on UxPlay latest 1.16.1, airplaying from the youtube in-app casting button works - only audio (previously, youtube app would hang). Seems to work as far back as Uxplay 1.53, so most likely a change on youtube's end.