search

FDH2 / UxPlay

AirPlay Unix mirroring server
GNU General Public License v3.0
1.55k stars 78 forks source link

still problems with e.g. 3rd party windows mirrroring software, as airmypc #25

Closed arararar closed 2 years ago

arararar commented 2 years ago

I think its a problem in the underlying sources (https://github.com/FD-/RPiPlay/issues/27) due to <ios9-implementation in the 3rd party software.

I am trying to offer my school the possibilty to both mirror ipads and surfaces wirelessly to our linux desktop pcs. The ipads are working flawlessly with uxplay. But the surfaces / windows notebooks are not working with e.g. "airmypc", i am getting the message:

... raotp_rtp_mirror.c:327: raop_rtp_mirror_thread: Assertion `nc_len > 0'

Is it possible to fix this problem?

arararar commented 2 years ago

i commented out line 327 to get rid of the assertion error (hope dies last), and heres my debuglog of this failed run:

`uxplay -n nono -d using randomly-generated MAC address 4e:1b:f6:98:4d:00

Initialized server socket(s) Accepted IPv4 client on socket 24 Local: 10.1.3.46 Remote: 10.1.3.59 Open connections: 1 httpd receiving on socket 24 conn_request Handling request GET with URL /info INFO len = -209725364 httpd receiving on socket 24 conn_request Handling request POST with URL /pair-setup httpd receiving on socket 24 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 24 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 24 conn_request Handling request POST with URL /fp-setup httpd receiving on socket 24 conn_request Handling request POST with URL /fp-setup httpd receiving on socket 24 conn_request Handling request SETUP with URL rtsp://10.1.3.46/2721820614363984414 DACP-ID: BAADABA995AF842F Active-Remote: 8312615868 Transport: null SETUP 1 eiv_len = 16 ekey_len = 72 fairplay_decrypt ret = 0 timing_rport = 51904 raop_ntp parse remote ip = 10.1.3.59 raop_ntp starting time raop_ntp local timing port socket 27 port UDP 60227 raop_rtp parse remote ip = 10.1.3.59 raop_rtp_mirror parse remote ip = 10.1.3.59 eport = 33849, tport = 60227 type = 110 streamConnectionID = 2721820614363984414 raop_rtp_mirror starting mirroring raop_rtp_mirror local data port socket 28 port TCP 33095 raop_ntp send_len = 32 Mirroring initialized successfully raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = -3843672980376750 Accepted IPv4 client on socket 29 Local: 10.1.3.46 Remote: 10.1.3.59 Open connections: 2 httpd receiving on socket 24 conn_request Handling request RECORD with URL rtsp://10.1.3.46/2721820614363984414 raop_handler_record httpd receiving on socket 24 conn_request Handling request GET with URL /info INFO len = -209725364 raop_rtp_mirror accepting client raop_rtp_mirror width_source = 1260.000000 height_source = 708.000000 width = 1795.000000 height = 1008.000000 raop_rtp_mirror sps size = 16 raop_rtp_mirror pps size = 4 raop_rtp_mirror video ntp = 3843672981277991, now = 1634684181291950, latency = -2208988799986041 raop_rtp_mirror video ntp = 3843672982392932, now = 1634684182427151, latency = -2208988799965781 httpd receiving on socket 24 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_rtp_mirror video ntp = 3843672983500151, now = 1634684183510251, latency = -2208988799989900 Speicherzugriffsfehler (Speicherabzug geschrieben) `

fduncanh commented 2 years ago

You got past the handshake part of the AirPlay protocol, so there does not seem to be any issue about the pre ios9 original AirPlay1 protocol, as FD- had speculated.

your error translates as Memory access error (memory dumped)

what crashed, UxPlay or AirMyPC?

are you using audio? try UxPlay -a to suppress it, and see if just video works.

stackrainbow commented 2 years ago

I had a look at this and I think what happens is the decryption fails and the while loop here locks up the thread (https://github.com/FDH2/UxPlay/blob/master/lib/raop_rtp_mirror.c#L324) or segfaults with the assert. I commented out the while loop, which stops the thread lockup and shows there is rapid inbound encrypted H264 data, but looking at the stream it looks like junk because the decryption process fails. I'm not entirely sure as to why the decryption fails, maybe the key negotiation process fails or is incomplete? Or maybe they are using an older version of the protocol which tries a different way.

arararar commented 2 years ago

Hi again! I retried to mirror win 10 notebook via airmypc / airparrot3 to uxplay server on debian linux.

linux uxplay server always generates new random mac which seems to disturb ipad from second relaunch on forever. would be cool to have an additional option to suppress the random mac or to provide a static one instead.

but back to topic: i tried airmypc with/without audio. But result is the same crash "len 0". It would be very kind if someone could look into this matter. Uxplay would be so much more useful, if windowsclients would be possible, too.

--edit This time i used the latest version, 1.41

--edit2 I also tested to mirror the win 10 notebook via airmypc to an cheap "roku express" stick. "airplay" in this case worked flawlessly, including sound transmission.

fduncanh commented 2 years ago

uxplay is not supposed to generate a random mac on linux unless the -m option is used, but I noticed it does on ubuntu. (It defaults to random mac only if it cant find the true mac) I'll check out why and fix it (It generally will use random mac on macOS by default)

Which windows clients have you tried (just airmypc?) OK I see airparrot3 too. Have you tried the latest uxplay?

Turns out there was a longstanding bug in the decryption on RPiplay (UxPlay parent) that I discovered. This is now fixed and the Airplay audio-only protocol (ALAC, Apple Lossless) now also works. Please retest with current UxPlay 1.41 just in case that was the issue.

arararar commented 2 years ago

I have noticed this random generated macs in ubuntu 21.10 and debian.

I tried airmypc and airparrot3. Both work with the airplay Implementation of the "roku express" stick, but not with uxplay.

I used uxplay version 1.41.

arararar commented 2 years ago

Here is my debug data: first run audio+video in server, second run audio only. both times i triggered "cast video only in airmypc".

`DEBUG Info uxplay 1.41 on Debian buster

./uxplay -p -n test4 Message: 10:55:41.895: supported audio format 1: AAC-ELD 44100/2 Message: 10:55:41.896: supported audio format 2: ALAC 44100/16/2 Message: 10:55:41.897: supported audio format 3: AAC-LC 44100/2 Message: 10:55:41.898: supported audio format 4: PCM 44100/16/2 S16LE using network ports UDP 7011 6001 6000 TCP 7100 7000 7001

using randomly-generated MAC address 16:72:68:26:05:6f

Initialized server socket(s) Accepted IPv4 client on socket 34 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 1 Connection closed for socket 34 Destroying connection Open connections: 0 Accepted IPv4 client on socket 34 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 1 ct=2 spf=352 usingScreen=1 isMedia=0 audioFormat=0x40000 ** Message: 10:55:57.099: start audio connection, format ALAC 44100/16/2 raop_rtp starting audio Accepted IPv4 client on socket 45 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 2 raop_rtp_mirror starting mirroring uxplay: /home/hsg/UxPlay/lib/raop_rtp_mirror.c:327: raop_rtp_mirror_thread: Assertion `nc_len > 0' failed. Abgebrochen

./uxplay -p -as 0 -n test5 audio_disabled using network ports UDP 7011 6001 6000 TCP 7100 7000 7001

using randomly-generated MAC address da:5b:32:1a:95:4e

Initialized server socket(s) Accepted IPv4 client on socket 26 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 1 raop_rtp_mirror starting mirroring Accepted IPv4 client on socket 29 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 2 uxplay: /home/hsg/UxPlay/lib/raop_rtp_mirror.c:327: raop_rtp_mirror_thread: Assertion `nc_len > 0' failed. Abgebrochen

`

arararar commented 2 years ago

And in addition, audioless transmission with "-d" flag enabled:

./uxplay -p -as 0 -d -n test6 audio_disabled using network ports UDP 7011 6001 6000 TCP 7100 7000 7001

using randomly-generated MAC address d6:ce:4b:47:c9:65

Initialized server socket(s) Accepted IPv4 client on socket 26 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 1 httpd receiving on socket 26 conn_request Handling request GET with URL /info INFO len = 763336644 httpd receiving on socket 26 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 26 conn_request Handling request POST with URL /pair-setup httpd receiving on socket 26 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 26 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 26 conn_request Handling request POST with URL /fp-setup httpd receiving on socket 26 conn_request Handling request POST with URL /fp-setup httpd receiving on socket 26 conn_request Handling request SETUP with URL rtsp://10.10.100.4/7420041104247548497 DACP-ID: 7D338352CFAF9489 Active-Remote: 7984906822 Transport: null <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

deviceID 0A:0A:64:04:10:10 eiv 5281+o1Ec1HOq+XjzNLLuA== ekey RlBMWQECAQAAAAA8AAAAAAAr+NgngR++RgN/BpVpSqgAAAAQ3pbas6EZIHzkwfzn0ean XhFFADDNS2ku9LQfn2LkdmYwyCp1 et 32 macAddress 0A:0A:64:04:10:10 model AirMyPC name AirMyPC osBuildVersion 13F69 sessionUUID AFE0A815-1EC4-4F0C-8F44-A36170EA4B71 sourceVersion 280.33 streams streamConnectionID 7420041104247548497 timestampInfo name SubSu name BePxT name AfPxT name BefEn name EmEnc type 110 timingPort 57540

SETUP 1 eiv_len = 16 ekey_len = 72 fairplay_decrypt ret = 0 timing_rport = 57540 raop_ntp parse remote ip = 10.10.238.128 raop_ntp starting time raop_ntp local timing port socket 27 port UDP 7011 raop_rtp parse remote ip = 10.10.238.128 raop_rtp_mirror parse remote ip = 10.10.238.128 eport = 7000, tport = 7011 type = 110 streamConnectionID = 7420041104247548497 raop_ntp send_len = 32 raop_rtp_mirror starting mirroring raop_rtp_mirror local data port socket 28 port TCP 7100 Mirroring initialized successfully raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = -3846045869930262 Accepted IPv4 client on socket 29 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 2 httpd receiving on socket 26 conn_request Handling request RECORD with URL rtsp://10.10.100.4/7420041104247548497 raop_handler_record httpd receiving on socket 26 conn_request Handling request GET with URL /info INFO len = 763336644 raop_rtp_mirror accepting client raop_rtp_mirror width_source = 1260.000000 height_source = 708.000000 width = 1795.000000 height = 1008.000000 raop_rtp_mirror sps size = 16 raop_rtp_mirror pps size = 4 raop_rtp_mirror video ntp = 3846045871505793, now = 1637057071541231, latency = -2208988799964562 httpd receiving on socket 26 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_rtp_mirror video ntp = 3846045872606490, now = 1637057072610079, latency = -2208988799996411 raop_ntp send_len = 32 raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = -1893 raop_rtp_mirror video ntp = 3846045873720446, now = 1637057073721371, latency = -2208988799999075 httpd receiving on socket 26 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_rtp_mirror video ntp = 3846045874824079, now = 1637057074826684, latency = -2208988799997395 uxplay: /home/hsg/UxPlay/lib/raop_rtp_mirror.c:327: raop_rtp_mirror_thread: Assertion `nc_len > 0' failed. Abgebrochen

arararar commented 2 years ago

If i cast "audio + video" with airmypc, audio dont seem tobe the problem, sth is wrong with video. btw, ipad - screencast works flawlessly in this setup, "audio+video" correct.

arararar commented 2 years ago

Plz dont be angry with me - but this run could provide precious debug data for someone who knows it better than me:

./uxplay -p -d -vs xvimagesink -n test11 Message: 11:14:25.322: supported audio format 1: AAC-ELD 44100/2 Message: 11:14:25.324: supported audio format 2: ALAC 44100/16/2 Message: 11:14:25.325: supported audio format 3: AAC-LC 44100/2 Message: 11:14:25.326: supported audio format 4: PCM 44100/16/2 S16LE using network ports UDP 7011 6001 6000 TCP 7100 7000 7001

using randomly-generated MAC address e6:7a:1c:69:c2:5f

Initialized server socket(s) Accepted IPv4 client on socket 29 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 1 httpd receiving on socket 29 conn_request Handling request GET with URL /info INFO len = -848185404 httpd receiving on socket 29 Connection closed for socket 29 Destroying connection Open connections: 0 Accepted IPv4 client on socket 29 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 1 httpd receiving on socket 29 conn_request Handling request GET with URL /info INFO len = -848185404 httpd receiving on socket 29 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 29 conn_request Handling request POST with URL /pair-setup httpd receiving on socket 29 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 29 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 29 conn_request Handling request POST with URL /fp-setup httpd receiving on socket 29 conn_request Handling request POST with URL /fp-setup httpd receiving on socket 29 conn_request Handling request SETUP with URL rtsp://10.10.100.4/6549601705745152332 DACP-ID: AF3AD2BCCCEC7317 Active-Remote: 1882964716 Transport: null <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

deviceID 0A:0A:64:04:10:10 eiv v/t51G5Uv7o8tAzb8kvP1w== ekey RlBMWQECAQAAAAA8AAAAAABeDjcQlPon3QnD4IVLfAMAAAAQihitVVwYeuUwXvclWETP +LsEFuKK9E2NdY3EPD7rorXjJPYR et 32 macAddress 0A:0A:64:04:10:10 model AirMyPC name AirMyPC osBuildVersion 13F69 sessionUUID D511BCB0-59AF-4693-98AD-780CCAA93807 sourceVersion 280.33 streams audioFormat 262144 controlPort 65532 ct 2 latencyMax 3750 latencyMin 3750 redundantAudio 2 spf 352 streamConnectionID 6549601705745152332 type 96 usingScreen timingPort 65531

SETUP 1 eiv_len = 16 ekey_len = 72 fairplay_decrypt ret = 0 timing_rport = 65531 raop_ntp parse remote ip = 10.10.238.128 raop_ntp starting time raop_ntp local timing port socket 30 port UDP 7011 raop_rtp parse remote ip = 10.10.238.128 raop_rtp_mirror parse remote ip = 10.10.238.128 eport = 7000, tport = 7011 type = 96 ct=2 spf=352 usingScreen=1 isMedia=0 audioFormat=0x40000 raop_ntp send_len = 32 ** Message: 11:14:30.853: start audio connection, format ALAC 44100/16/2 raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = -3846046470257403 raop_rtp starting audio raop_rtp local control port socket 38 port UDP 6001 raop_rtp local data port socket 39 port UDP 6000 RAOP initialized success Accepted IPv4 client on socket 40 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 2 httpd receiving on socket 29 conn_request Handling request RECORD with URL rtsp://10.10.100.4/6549601705745152332 raop_handler_record httpd receiving on socket 29 conn_request Handling request GET with URL /info INFO len = -848185404 httpd receiving on socket 29 conn_request Handling request SET_PARAMETER with URL rtsp://10.10.100.4/6549601705745152332 httpd receiving on socket 29 conn_request Handling request SET_PARAMETER with URL rtsp://10.10.100.4/6549601705745152332 httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_ntp send_len = 32 raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = 0 httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_ntp send_len = 32 raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = 0 httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_ntp send_len = 32 raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = 0 httpd receiving on socket 29 conn_request Handling request SETUP with URL rtsp://10.10.100.4/2348515325343002466 DACP-ID: AF3AD2BCCCEC7317 Active-Remote: 1882964716 Transport: null <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

deviceID 0A:0A:64:04:10:10 macAddress 0A:0A:64:04:10:10 model AirMyPC name AirMyPC osBuildVersion 13F69 sessionUUID D511BCB0-59AF-4693-98AD-780CCAA93807 sourceVersion 280.33 streams streamConnectionID 2348515325343002466 timestampInfo name SubSu name BePxT name AfPxT name BefEn name EmEnc type 110

type = 110 streamConnectionID = 2348515325343002466 raop_rtp_mirror starting mirroring raop_rtp_mirror local data port socket 41 port TCP 7100 Mirroring initialized successfully raop_rtp_mirror accepting client raop_rtp audio: ntp = 0, now = 1637057681407425, latency=1637057681407425, rtp=0 httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_rtp_mirror width_source = 1280.000000 height_source = 720.000000 width = 1823.000000 height = 1025.000000 raop_rtp_mirror sps size = 15 raop_rtp_mirror pps size = 4 raop_rtp_mirror video ntp = 3846046481727699, now = 1637057681813544, latency = -2208988799914155 raop_rtp_mirror video ntp = 3846046482871251, now = 1637057682938451, latency = -2208988799932800 GStreamer error: Error parsing H.264 stream raop_ntp send_len = 32 Re-launching server... raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = 0 Removing connection for socket 29 Destroying connection Open connections: 1 raop_ntp stopping time thread raop_ntp exiting thread raop_ntp stopped time thread raop_rtp exiting thread raop_rtp_mirror exiting TCP thread Removing connection for socket 40 Destroying connection Open connections: 0 Exiting HTTP thread Initialized server socket(s) Accepted IPv4 client on socket 29 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 1 httpd receiving on socket 29 conn_request Handling request GET with URL /info INFO len = -848185404 httpd receiving on socket 29 Connection closed for socket 29 Destroying connection Open connections: 0 Accepted IPv4 client on socket 29 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 1 httpd receiving on socket 29 conn_request Handling request GET with URL /info INFO len = -848185404 httpd receiving on socket 29 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 29 conn_request Handling request POST with URL /pair-setup httpd receiving on socket 29 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 29 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 29 conn_request Handling request POST with URL /fp-setup httpd receiving on socket 29 conn_request Handling request POST with URL /fp-setup httpd receiving on socket 29 conn_request Handling request SETUP with URL rtsp://10.10.100.4/3963343491738096178 DACP-ID: A5ACBB4DFF666845 Active-Remote: 9103267054 Transport: null <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

deviceID 0A:0A:64:04:10:10 eiv FPG3jy7QNxrdcFhY++oqeg== ekey RlBMWQECAQAAAAA8AAAAAABZUPGAxebe29QOaUKQgoEAAAAQrLxlyup8WL96K2rTovVO qYtfra09TyA6u11GF0YeNQkb62ct et 32 macAddress 0A:0A:64:04:10:10 model AirMyPC name AirMyPC osBuildVersion 13F69 sessionUUID EBD30B2A-1DA6-4028-AFE1-11B88850D9FB sourceVersion 280.33 streams audioFormat 262144 controlPort 59234 ct 2 latencyMax 3750 latencyMin 3750 redundantAudio 2 spf 352 streamConnectionID 3963343491738096178 type 96 usingScreen timingPort 59233

SETUP 1 eiv_len = 16 ekey_len = 72 fairplay_decrypt ret = 0 timing_rport = 59233 raop_ntp parse remote ip = 10.10.238.128 raop_ntp starting time raop_ntp local timing port socket 30 port UDP 7011 raop_rtp parse remote ip = 10.10.238.128 raop_rtp_mirror parse remote ip = 10.10.238.128 eport = 7000, tport = 7011 type = 96 ct=2 spf=352 usingScreen=1 isMedia=0 audioFormat=0x40000 raop_rtp starting audio raop_rtp local control port socket 38 port UDP 6001 raop_rtp local data port socket 39 port UDP 6000 raop_ntp send_len = 32 RAOP initialized success raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = -3846046502826615 Accepted IPv4 client on socket 40 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 2 httpd receiving on socket 29 conn_request Handling request RECORD with URL rtsp://10.10.100.4/3963343491738096178 raop_handler_record httpd receiving on socket 29 conn_request Handling request GET with URL /info INFO len = -848185404 httpd receiving on socket 29 conn_request Handling request SET_PARAMETER with URL rtsp://10.10.100.4/3963343491738096178 httpd receiving on socket 29 conn_request Handling request SET_PARAMETER with URL rtsp://10.10.100.4/3963343491738096178 httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_ntp send_len = 32 raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = 0 httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_ntp send_len = 32 raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = 0 httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_ntp send_len = 32 raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = 89 httpd receiving on socket 29 conn_request Handling request SETUP with URL rtsp://10.10.100.4/7032829555641182226 DACP-ID: A5ACBB4DFF666845 Active-Remote: 9103267054 Transport: null <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

deviceID 0A:0A:64:04:10:10 macAddress 0A:0A:64:04:10:10 model AirMyPC name AirMyPC osBuildVersion 13F69 sessionUUID EBD30B2A-1DA6-4028-AFE1-11B88850D9FB sourceVersion 280.33 streams streamConnectionID 7032829555641182226 timestampInfo name SubSu name BePxT name AfPxT name BefEn name EmEnc type 110

type = 110 streamConnectionID = 7032829555641182226 raop_rtp_mirror starting mirroring raop_rtp_mirror local data port socket 41 port TCP 7100 Mirroring initialized successfully raop_rtp_mirror accepting client raop_rtp audio: ntp = 0, now = 1637057714089532, latency=1637057714089532, rtp=0 raop_rtp_mirror width_source = 1280.000000 height_source = 720.000000 width = 1823.000000 height = 1025.000000 raop_rtp_mirror sps size = 15 raop_rtp_mirror pps size = 4 httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_rtp_mirror video ntp = 3846046514309590, now = 1637057714370828, latency = -2208988799938762 uxplay: /home/hsg/UxPlay/lib/raop_rtp_mirror.c:327: raop_rtp_mirror_thread: Assertion `nc_len > 0' failed. Abgebrochen

./uxplay -p -d -vs xvimagesink -n test11 Message: 11:14:25.322: supported audio format 1: AAC-ELD 44100/2 Message: 11:14:25.324: supported audio format 2: ALAC 44100/16/2 Message: 11:14:25.325: supported audio format 3: AAC-LC 44100/2 Message: 11:14:25.326: supported audio format 4: PCM 44100/16/2 S16LE using network ports UDP 7011 6001 6000 TCP 7100 7000 7001

using randomly-generated MAC address e6:7a:1c:69:c2:5f

Initialized server socket(s) Accepted IPv4 client on socket 29 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 1 httpd receiving on socket 29 conn_request Handling request GET with URL /info INFO len = -848185404 httpd receiving on socket 29 Connection closed for socket 29 Destroying connection Open connections: 0 Accepted IPv4 client on socket 29 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 1 httpd receiving on socket 29 conn_request Handling request GET with URL /info INFO len = -848185404 httpd receiving on socket 29 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 29 conn_request Handling request POST with URL /pair-setup httpd receiving on socket 29 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 29 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 29 conn_request Handling request POST with URL /fp-setup httpd receiving on socket 29 conn_request Handling request POST with URL /fp-setup httpd receiving on socket 29 conn_request Handling request SETUP with URL rtsp://10.10.100.4/6549601705745152332 DACP-ID: AF3AD2BCCCEC7317 Active-Remote: 1882964716 Transport: null <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

deviceID 0A:0A:64:04:10:10 eiv v/t51G5Uv7o8tAzb8kvP1w== ekey RlBMWQECAQAAAAA8AAAAAABeDjcQlPon3QnD4IVLfAMAAAAQihitVVwYeuUwXvclWETP +LsEFuKK9E2NdY3EPD7rorXjJPYR et 32 macAddress 0A:0A:64:04:10:10 model AirMyPC name AirMyPC osBuildVersion 13F69 sessionUUID D511BCB0-59AF-4693-98AD-780CCAA93807 sourceVersion 280.33 streams audioFormat 262144 controlPort 65532 ct 2 latencyMax 3750 latencyMin 3750 redundantAudio 2 spf 352 streamConnectionID 6549601705745152332 type 96 usingScreen timingPort 65531

SETUP 1 eiv_len = 16 ekey_len = 72 fairplay_decrypt ret = 0 timing_rport = 65531 raop_ntp parse remote ip = 10.10.238.128 raop_ntp starting time raop_ntp local timing port socket 30 port UDP 7011 raop_rtp parse remote ip = 10.10.238.128 raop_rtp_mirror parse remote ip = 10.10.238.128 eport = 7000, tport = 7011 type = 96 ct=2 spf=352 usingScreen=1 isMedia=0 audioFormat=0x40000 raop_ntp send_len = 32 ** Message: 11:14:30.853: start audio connection, format ALAC 44100/16/2 raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = -3846046470257403 raop_rtp starting audio raop_rtp local control port socket 38 port UDP 6001 raop_rtp local data port socket 39 port UDP 6000 RAOP initialized success Accepted IPv4 client on socket 40 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 2 httpd receiving on socket 29 conn_request Handling request RECORD with URL rtsp://10.10.100.4/6549601705745152332 raop_handler_record httpd receiving on socket 29 conn_request Handling request GET with URL /info INFO len = -848185404 httpd receiving on socket 29 conn_request Handling request SET_PARAMETER with URL rtsp://10.10.100.4/6549601705745152332 httpd receiving on socket 29 conn_request Handling request SET_PARAMETER with URL rtsp://10.10.100.4/6549601705745152332 httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_ntp send_len = 32 raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = 0 httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_ntp send_len = 32 raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = 0 httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_ntp send_len = 32 raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = 0 httpd receiving on socket 29 conn_request Handling request SETUP with URL rtsp://10.10.100.4/2348515325343002466 DACP-ID: AF3AD2BCCCEC7317 Active-Remote: 1882964716 Transport: null <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

deviceID 0A:0A:64:04:10:10 macAddress 0A:0A:64:04:10:10 model AirMyPC name AirMyPC osBuildVersion 13F69 sessionUUID D511BCB0-59AF-4693-98AD-780CCAA93807 sourceVersion 280.33 streams streamConnectionID 2348515325343002466 timestampInfo name SubSu name BePxT name AfPxT name BefEn name EmEnc type 110

type = 110 streamConnectionID = 2348515325343002466 raop_rtp_mirror starting mirroring raop_rtp_mirror local data port socket 41 port TCP 7100 Mirroring initialized successfully raop_rtp_mirror accepting client raop_rtp audio: ntp = 0, now = 1637057681407425, latency=1637057681407425, rtp=0 httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_rtp_mirror width_source = 1280.000000 height_source = 720.000000 width = 1823.000000 height = 1025.000000 raop_rtp_mirror sps size = 15 raop_rtp_mirror pps size = 4 raop_rtp_mirror video ntp = 3846046481727699, now = 1637057681813544, latency = -2208988799914155 raop_rtp_mirror video ntp = 3846046482871251, now = 1637057682938451, latency = -2208988799932800 GStreamer error: Error parsing H.264 stream raop_ntp send_len = 32 Re-launching server... raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = 0 Removing connection for socket 29 Destroying connection Open connections: 1 raop_ntp stopping time thread raop_ntp exiting thread raop_ntp stopped time thread raop_rtp exiting thread raop_rtp_mirror exiting TCP thread Removing connection for socket 40 Destroying connection Open connections: 0 Exiting HTTP thread Initialized server socket(s) Accepted IPv4 client on socket 29 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 1 httpd receiving on socket 29 conn_request Handling request GET with URL /info INFO len = -848185404 httpd receiving on socket 29 Connection closed for socket 29 Destroying connection Open connections: 0 Accepted IPv4 client on socket 29 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 1 httpd receiving on socket 29 conn_request Handling request GET with URL /info INFO len = -848185404 httpd receiving on socket 29 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 29 conn_request Handling request POST with URL /pair-setup httpd receiving on socket 29 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 29 conn_request Handling request POST with URL /pair-verify httpd receiving on socket 29 conn_request Handling request POST with URL /fp-setup httpd receiving on socket 29 conn_request Handling request POST with URL /fp-setup httpd receiving on socket 29 conn_request Handling request SETUP with URL rtsp://10.10.100.4/3963343491738096178 DACP-ID: A5ACBB4DFF666845 Active-Remote: 9103267054 Transport: null <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

deviceID 0A:0A:64:04:10:10 eiv FPG3jy7QNxrdcFhY++oqeg== ekey RlBMWQECAQAAAAA8AAAAAABZUPGAxebe29QOaUKQgoEAAAAQrLxlyup8WL96K2rTovVO qYtfra09TyA6u11GF0YeNQkb62ct et 32 macAddress 0A:0A:64:04:10:10 model AirMyPC name AirMyPC osBuildVersion 13F69 sessionUUID EBD30B2A-1DA6-4028-AFE1-11B88850D9FB sourceVersion 280.33 streams audioFormat 262144 controlPort 59234 ct 2 latencyMax 3750 latencyMin 3750 redundantAudio 2 spf 352 streamConnectionID 3963343491738096178 type 96 usingScreen timingPort 59233

SETUP 1 eiv_len = 16 ekey_len = 72 fairplay_decrypt ret = 0 timing_rport = 59233 raop_ntp parse remote ip = 10.10.238.128 raop_ntp starting time raop_ntp local timing port socket 30 port UDP 7011 raop_rtp parse remote ip = 10.10.238.128 raop_rtp_mirror parse remote ip = 10.10.238.128 eport = 7000, tport = 7011 type = 96 ct=2 spf=352 usingScreen=1 isMedia=0 audioFormat=0x40000 raop_rtp starting audio raop_rtp local control port socket 38 port UDP 6001 raop_rtp local data port socket 39 port UDP 6000 raop_ntp send_len = 32 RAOP initialized success raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = -3846046502826615 Accepted IPv4 client on socket 40 Local: 10.10.100.4 Remote: 10.10.238.128 Open connections: 2 httpd receiving on socket 29 conn_request Handling request RECORD with URL rtsp://10.10.100.4/3963343491738096178 raop_handler_record httpd receiving on socket 29 conn_request Handling request GET with URL /info INFO len = -848185404 httpd receiving on socket 29 conn_request Handling request SET_PARAMETER with URL rtsp://10.10.100.4/3963343491738096178 httpd receiving on socket 29 conn_request Handling request SET_PARAMETER with URL rtsp://10.10.100.4/3963343491738096178 httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_ntp send_len = 32 raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = 0 httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_ntp send_len = 32 raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = 0 httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_ntp send_len = 32 raop_ntp receive time type_t packetlen = 32 raop_ntp sync correction = 89 httpd receiving on socket 29 conn_request Handling request SETUP with URL rtsp://10.10.100.4/7032829555641182226 DACP-ID: A5ACBB4DFF666845 Active-Remote: 9103267054 Transport: null <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

deviceID 0A:0A:64:04:10:10 macAddress 0A:0A:64:04:10:10 model AirMyPC name AirMyPC osBuildVersion 13F69 sessionUUID EBD30B2A-1DA6-4028-AFE1-11B88850D9FB sourceVersion 280.33 streams streamConnectionID 7032829555641182226 timestampInfo name SubSu name BePxT name AfPxT name BefEn name EmEnc type 110

type = 110 streamConnectionID = 7032829555641182226 raop_rtp_mirror starting mirroring raop_rtp_mirror local data port socket 41 port TCP 7100 Mirroring initialized successfully raop_rtp_mirror accepting client raop_rtp audio: ntp = 0, now = 1637057714089532, latency=1637057714089532, rtp=0 raop_rtp_mirror width_source = 1280.000000 height_source = 720.000000 width = 1823.000000 height = 1025.000000 raop_rtp_mirror sps size = 15 raop_rtp_mirror pps size = 4 httpd receiving on socket 29 conn_request Handling request POST with URL /feedback raop_handler_feedback raop_rtp_mirror video ntp = 3846046514309590, now = 1637057714370828, latency = -2208988799938762 uxplay: /home/hsg/UxPlay/lib/raop_rtp_mirror.c:327: raop_rtp_mirror_thread: Assertion `nc_len > 0' failed. Abgebrochen

fduncanh commented 2 years ago

Does audio-only (AirPlay, not Airplay mirror, from Apple Music_ work at all ?)

fduncanh commented 2 years ago

As you noted the issue is entirely in raop_rtp_mirror.c

               // It seems the AirPlay protocol prepends NALs with their size, which we're replacing with the 4-byte
                // start code for the NAL Byte-Stream Format.
                while (nalu_size < payload_size) {
                    int nc_len = (payload_decrypted[nalu_size + 0] << 24) | (payload_decrypted[nalu_size + 1] << 16) |
                                 (payload_decrypted[nalu_size + 2] << 8) | (payload_decrypted[nalu_size + 3]);
                    assert(nc_len > 0);

                    payload_decrypted[nalu_size + 0] = 0;
                    payload_decrypted[nalu_size + 1] = 0;
                    payload_decrypted[nalu_size + 2] = 0;
                    payload_decrypted[nalu_size + 3] = 1;
                    nalu_size += nc_len + 4;
                    nalus_count++;
                }

                // logger_log(raop_rtp_mirror->logger, LOGGER_DEBUG, "nalutype = %d", nalu_type);
                // logger_log(raop_rtp_mirror->logger, LOGGER_DEBUG, "nalu_size = %d, payloadsize = %d nalus_count = %d",
                //        nalu_size, payload_size, nalus_count);

Unfortunately, I dont have a working (or partially working) AirMyPc of AirParrot to play with. I installed AirParrot on a windows desktop (with a wired ethernet connection) and it saw Uxplay on the network, but nothing happened on UxPlay when I tried to start a connection. Putting in printf statements to examine payload_decrypted with AirMyPc and with ipad would be needed to see what nc_len and payload_decrypted contain

fduncanh commented 2 years ago

OK what seems to be going wrong is the AES-CTR decryption.

#ifdef DUMP_H264
                fwrite(payload, payload_size, 1, file_source);
                fwrite(&readstart, sizeof(readstart), 1, file_len);
#endif

                // Decrypt data
                unsigned char* payload_decrypted = malloc(payload_size);
                mirror_buffer_decrypt(raop_rtp_mirror->buffer, payload, payload_decrypted, payload_size);

                //int nalu_type = payload[4] & 0x1f;
                int nalu_size = 0;
                int nalus_count = 0;

                // It seems the AirPlay protocol prepends NALs with their size, which we're replacing with the 4-byte
                // start code for the NAL Byte-Stream Format.
                while (nalu_size < payload_size) {
                    int nc_len = (payload_decrypted[nalu_size + 0] << 24) | (payload_decrypted[nalu_size + 1] << 16) |
                                 (payload_decrypted[nalu_size + 2] << 8) | (payload_decrypted[nalu_size + 3]);
                    assert(nc_len > 0);

                    payload_decrypted[nalu_size + 0] = 0;
                    payload_decrypted[nalu_size + 1] = 0;
                    payload_decrypted[nalu_size + 2] = 0;
                    payload_decrypted[nalu_size + 3] = 1;
                    nalu_size += nc_len + 4;
                    nalus_count++;
                }

                // logger_log(raop_rtp_mirror->logger, LOGGER_DEBUG, "nalutype = %d", nalu_type);
                // logger_log(raop_rtp_mirror->logger, LOGGER_DEBUG, "nalu_size = %d, payloadsize = %d nalus_count = %d",
                //        nalu_size, payload_size, nalus_count);

#ifdef DUMP_H264
                fwrite(payload_decrypted, payload_size, 1, file);
#endif

so we need to look at the packets

fduncanh commented 2 years ago

inspect the video packets by replacing line 316 of lib/raop_rtp_mirror.c (sandwiching it) by 

                printf("encrypted packet: %d %d\n",payload_size, payload_size%16);
                for (int i = 0; i < payload_size; i++ ) {
                    if (i == 32) printf("...\n");
                    if( i > 31 && i < (payload_size/16*16) - 32)  continue;
                    printf("%2.2x", *(payload+i));
                    if(i %16 == 15) printf("\n");
                }
                printf("\n");

                mirror_buffer_decrypt(raop_rtp_mirror->buffer, payload, payload_decrypted, payload_size);

                printf("\ndecrypted packet: %d\n",payload_size);
                for (int i = 0; i < payload_size; i++ ) {
                    if (i == 32) printf("...\n");
                    if ( i > 31 && i < (payload_size/16*16) - 32)  continue;
                    printf("%2.2x", *(payload_decrypted+i));
                   if(i %16 == 15) printf("\n");
                }
                printf("\n");
                for ( int i = 0; i < 32 ; i++) printf("-");
                printf("\n");

post an example packet that works (from iPad) and one that fails (from airMyPC/airparrot).

run 

uxplay     > out

and look at the top of the file "out" to get the initial video packets. don't use -d option any more, it won't help for this problem, I think.

arararar commented 2 years ago

THANK YOU VERY MUCH FOR LOOKING into this matter - and your guidance. I will do so in est. 3 hours, kids need to stay in bed till then :)

fduncanh commented 2 years ago

Here are my first two (working) video packets:

using network ports UDP 7011 6001 6000 TCP 7100 7000 7001
using system MAC address xx:xx:xx:xx:xx:xx
Initialized server socket(s)
Accepted IPv4 client on socket 30
Local: 192.168.1.8
Remote: 192.168.1.105
Open connections: 1
Accepted IPv4 client on socket 32
Local: 192.168.1.8
Remote: 192.168.1.105
Open connections: 2
raop_rtp_mirror starting mirroring
encrypted packet: 14330 10
715636de196e4a09df4cba3a57e948dc
9d096d41405f79d43bd1b96ccc52386e
...
45c2ba310197679a7e1b436bb40469f4
fd6a5b025fdf714393f3347dde30b1f3
9ca80c57632aa57f89ea

decrypted packet: 14330
000037f625b8004000ffde08e3c6ff82
bbdd50d1f683593194b33f1de4b67a51
...
a22c217b6097a097b09b97676ec634db
04ec168429763a33769567e9a1d69ffb
09a92c049654426646c0
--------------------------------
encrypted packet: 401 1
1017edbb44ce5976d037a53e1859966a
bd4f1bdbc245aab90ee3744c1ecaa3ed
...
4b4c4988bb2156bb89ede77cdb107b87
ee66cab255bb16f18fafa043c671cf4e
ba

decrypted packet: 401
0000018d21e0020012567588ffd65572
b04aaef4e697b2c23e194b9fb9efeb06
...
27d9978c79dbfef36fb2e971e18e6d2a
11e4c6bdd00bcba6a2dd589ebd08fbea
ab
--------------------------------
fduncanh commented 2 years ago

Random MACs are now fixed, at least.

fduncanh commented 2 years ago

The aes-ctr decryption of packets seem a bit strange in mirror_buffer.c (mirror_buffer_decrypt) you can read about the decryption protocol (AES-CTR for video, AES-CBC for audio in the two modern accounts mentioned at the end of the first paragraph of the README:

Maybe some shortcuts in decryption are being made that take advantage of some iOS feature that is not completely replicated by airMyPC etc.

arararar commented 2 years ago

Here the working example from ipad, uxplay on Debian bullseye: (i will post the airmypc-run tomorrow, i have to fetch my windows notebook from work for that)

./uxplay


using system MAC address 80:fa:5b:08:7b:b9     <-------------   MAC address issue is now fixed
Initialized server socket(s)
Accepted IPv4 client on socket 27
Local: 192.168.0.181
Remote: 192.168.0.142
Open connections: 1
Accepted IPv4 client on socket 29
Local: 192.168.0.181
Remote: 192.168.0.142
Open connections: 2
raop_rtp_mirror starting mirroring
202f22c92baa9e1394cc9afeafc1ae17
03f5325573b92f012dd88d3aa7d29de5
...
20f0797d1104c82150a08599cccb6c2d
dbd1959e926e0feb9f0353c2dfb93cfa
7fe3d349265360233d565b5ca88d

decrypted packet: 12798
000031fa25b8004000ffde08a3c7ff05    <---- decrypted packet starts with 00 00
4deb731999257f031cf3d4f1edbda010
...
4e77ffe1fd693d19cd9d06d8bc4cd5d5
fefce7abaa98bdd39a5279db9a1fda00
0003000017a15a4e7c9206490dec
--------------------------------
8887bdea2ed28339c57797717f2987ce
5029b147b4f4e80bb69993b322640dac
...
5153d3142c1a3e4299e456d78d5c989a
32db9c3b00cad93bd19f55e9bf6ca6ef
dd7cc2f9cb1120308cdc863a4fc1

decrypted packet: 1118
0000045a21e0020012567593ffc97734    <---- decrypted packet starts with 00 00
24053ae05d4b631db4d6b0dc0450107f
...
db763512d475cc1cfcc2a94021803c30
1dd14d8e74b9700397954c2d7ca19c96
4dec1224d84c76c78d3ce89406d0
fduncanh commented 2 years ago

Its only the airmypc data that will reveal anything.

The above is OK data. I'm guessing that the decrypted packets on airmypc are corrupt, and we will find they dont start with two null bytes 00 00

In the above the first 4 bytes are packet 1: 00 00 31 fa
packet 2: 00 00 04 5a

I dont really understand the processing that happens after decryption: "nalu" = NAL unit NAL = Network Abstraction Layer https://en.wikipedia.org/wiki/Network_Abstraction_Layer

               //int nalu_type = payload[4] & 0x1f;
                int nalu_size = 0;
                int nalus_count = 0;

                // It seems the AirPlay protocol prepends NALs with their size, which we're replacing with the 4-byte
                // start code for the NAL Byte-Stream Format.
                while (nalu_size < payload_size) {
                    int nc_len = (payload_decrypted[nalu_size + 0] << 24) | (payload_decrypted[nalu_size + 1] << 16) |
                                 (payload_decrypted[nalu_size + 2] << 8) | (payload_decrypted[nalu_size + 3]);
                    assert(nc_len > 0);

                    payload_decrypted[nalu_size + 0] = 0;
                    payload_decrypted[nalu_size + 1] = 0;
                    payload_decrypted[nalu_size + 2] = 0;
                    payload_decrypted[nalu_size + 3] = 1;
                    nalu_size += nc_len + 4;
                    nalus_count++;
                }

                // logger_log(raop_rtp_mirror->logger, LOGGER_DEBUG, "nalutype = %d", nalu_type);
                // logger_log(raop_rtp_mirror->logger, LOGGER_DEBUG, "nalu_size = %d, payloadsize = %d nalus_count = %d",
                //        nalu_size, payload_size, nalus_count);
fduncanh commented 2 years ago

Pl;ease also test audio only with pure AirPlay (not AirPlay video). This will not decrypt any video packets.The sound is encrypted witrh a different method. I want to know it the audio is being decrypted correctly, or is Do AirMyPc aand ord airparrot do audio only.

I saw your output above ( a few posts back) has

deviceID 0A:0A:64:04:10:10 eiv FPG3jy7QNxrdcFhY++oqeg== ekey RlBMWQECAQAAAAA8AAAAAABZUPGAxebe29QOaUKQgoEAAAAQrLxlyup8WL96K2rTovVO qYtfra09TyA6u11GF0YeNQkb62ct et 32 macAddress 0A:0A:64:04:10:10 model AirMyPC name AirMyPC osBuildVersion 13F69 sessionUUID EBD30B2A-1DA6-4028-AFE1-11B88850D9FB sourceVersion 280.33 streams audioFormat 262144 controlPort 59234 ct 2 latencyMax 3750 latencyMin 3750 redundantAudio 2 spf 352 streamConnectionID 3963343491738096178 type 96 usingScreen timingPort 59233

SETUP 1
eiv_len = 16
ekey_len = 72
fairplay_decrypt ret = 0
timing_rport = 59233
raop_ntp parse remote ip = 10.10.238.128
raop_ntp starting time
raop_ntp local timing port socket 30 port UDP 7011
raop_rtp parse remote ip = 10.10.238.128
raop_rtp_mirror parse remote ip = 10.10.238.128
eport = 7000, tport = 7011
type = 96
ct=2 spf=352 usingScreen=1 isMedia=0 audioFormat=0x40000    <-- ct=2 is ALAC format pure audio (ct = 8 is audio with mirror)
raop_rtp starting audio

The plist data in the first line sent by airmypc seem to miss some line breaks between items.

the quantity streamConnectionID 3963343491738096178 is used in constructing the decryption key and IV (initial vector) if that is not being read properly, that could be the problem?

put this code after line 81 in lib/mirror_buffer.c, and run uxplay with the -d debug mode (which will show what the value of streamConnectionID sent by the clients is, as seen above).

The code below will show what value was actually used in mirror_buffer.c for the decryption process

printf ("streamConnectionID = %lu\n", streamConnectionID);
fduncanh commented 2 years ago

also, don't rely on the email you get, but look at the issue page on the UxPlay website, because I edit the comments, and maybe only the original unedited comment is sent to you by email

fduncanh commented 2 years ago

decrypted packet: 12798 00 00 31 fa 25b8004000ffde08a3c7ff05 <---- decrypted packet starts with 00 00

00 00 31 fa = 12794 = nc_len (a signed integer length) This the decrypted packet length (12798 -4)

decrypted packet: 1118 00 00 04 5a 21e0020012567593ffc97734 <---- decrypted packet starts with 00 00

00 00 04 5a = 1114 = nc_len This is the decrypted packet length 1118 -4) So the first 4 bytes that code nc_len are not part of the video frame, which consists of nc_len bytes. we can recognize good decryption

If the first digit is > 7 nc_len would be negative, that triggers the assert.

so as noted by @stackrainbow the airmypc decryption has failed. The question is why.

It relies on streamConnectionID aesiv aeskey. ecdh_secret.

fduncanh commented 2 years ago

Looking at your debug outputs it seems that it inits audio (type 96) first with format ct=2 (ALAC) before video (type 110) This is probably legacy AIrPlay1 mirror protocol.? It might or might not be easy to fix, but ....

Here we init video 110 first, then audio type 96, with ct=8 AAC-ELD audio format.

fduncanh commented 2 years ago

@arararar I download the mac version of airparrot3, but cant get it to get as far as you have. Are we talking about airparrot 3 or 2?

I see UxPlay and I see the "quick connect code" box on the airparrot panel. If I type in the ip address of the server (192.168.x.x)- as they say one should do if not using their reflector app as the receiver -- there is a small amount of activity from uxplay

conn_request
httpd didn't get response

But airparrot3 is trying to connect to 192.168.x.x not UxPlay

I would imagine that on widows something similar is supposed to happen.

How do you get as far as you do (or is that only on airmypc?).

On the original AirPlay mirror implementation, the video was unencrypted. I am wandering whether your video stream is unencrypted, in which case the decryption would trash it! a look at the beginnings of the before and after decryption packets that you said you would post might be interesting.

The fact that you get so far with the pairing between sender and receiver suggest much is correct.

arararar commented 2 years ago

@fduncanh i only use airmypc, because it can see / manually add uxplay server ip. Airparrot3 didnt work for me.

I failed to get the airmypc-sandwiched debug data via "> out". I think i will get around this problem soon.

fduncanh commented 2 years ago

assuming no firewall (-p ) settings,

uxplay -d > out 2>&1

I see airmypc reports "et = 32" for encryption type, same as iPad,

The exception that is triggered by detecting an invalid video stream occurs after a type 110 mirroring setup call.

conn_request
Handling request SETUP with URL rtsp://10.10.100.4/7032829555641182226
DACP-ID: A5ACBB4DFF666845
Active-Remote: 9103267054
Transport: null
deviceID 0A:0A:64:04:10:10 macAddress 0A:0A:64:04:10:10 model AirMyPC name AirMyPC osBuildVersion 13F69 sessionUUID EBD30B2A-1DA6-4028-AFE1-11B88850D9FB sourceVersion 280.33 streams streamConnectionID 7032829555641182226 timestampInfo name SubSu name BePxT name AfPxT name BefEn name EmEnc type 110

type = 110
streamConnectionID = 7032829555641182226
raop_rtp_mirror starting mirroring
raop_rtp_mirror local data port socket 41 port TCP 7100
Mirroring initialized successfully
raop_rtp_mirror accepting client
raop_rtp audio: ntp = 0, now = 1637057714089532, latency=1637057714089532, rtp=0
raop_rtp_mirror width_source = 1280.000000 height_source = 720.000000 width = 1823.000000 height = 1025.000000
raop_rtp_mirror sps size = 15
raop_rtp_mirror pps size = 4
httpd receiving on socket 29
conn_request
Handling request POST with URL /feedback
raop_handler_feedback
raop_rtp_mirror video ntp = 3846046514309590, now = 1637057714370828, latency = -2208988799938762
uxplay: /home/hsg/UxPlay/lib/raop_rtp_mirror.c:327: raop_rtp_mirror_thread: Assertion `nc_len > 0' failed.
Abgebrochen

If you can stream a pure audio track with myairpc only a type 96 Airplay setup will be made and type 110 (followed by Abgebrochen) will not occur. It would be interesting to know if the audio is decrypted properly.

As an alternative, let's just dump the video after (incorrectly) decrypting it by the following changes (so no mirror screen will open, but audio will be processed, so you should stream something like a music video from youtube to to see if it plays audio (using debug mode)

in raop_rtp_mirror.c, at lines 317 and 351, just let the bad decrypted video data to disappear into a "black hole" by enclosing it in if (0) { ..... }


                // Decrypt data
                unsigned char* payload_decrypted = malloc(payload_size);
                mirror_buffer_decrypt(raop_rtp_mirror->buffer, payload, payload_decrypted, payload_size);

               if (0 ) {                         <-------------------------------------------------added here after decrypt

                //int nalu_type = payload[4] & 0x1f;
                int nalu_size = 0;
                int nalus_count = 0;

                // It seems the AirPlay protocol prepends NALs with their size, which we're replacing with the 4-byte
                // start code for the NAL Byte-Stream Format.
                while (nalu_size < payload_size) {
                    int nc_len = (payload_decrypted[nalu_size + 0] << 24) | (payload_decrypted[nalu_size + 1] << 16) |
                                 (payload_decrypted[nalu_size + 2] << 8) | (payload_decrypted[nalu_size + 3]);
                    assert(nc_len > 0);

                    payload_decrypted[nalu_size + 0] = 0;
                    payload_decrypted[nalu_size + 1] = 0;
                    payload_decrypted[nalu_size + 2] = 0;
                    payload_decrypted[nalu_size + 3] = 1;
                    nalu_size += nc_len + 4;
                    nalus_count++;
                }

                // logger_log(raop_rtp_mirror->logger, LOGGER_DEBUG, "nalutype = %d", nalu_type);
                // logger_log(raop_rtp_mirror->logger, LOGGER_DEBUG, "nalu_size = %d, payloadsize = %d nalus_count = %d",
                //        nalu_size, payload_size, nalus_count);

#ifdef DUMP_H264
                fwrite(payload_decrypted, payload_size, 1, file);
#endif

                h264_decode_struct h264_data;
                h264_data.data_len = payload_size;
                h264_data.data = payload_decrypted;
                h264_data.frame_type = 1;
                h264_data.pts = ntp_timestamp;

                raop_rtp_mirror->callbacks.video_process(raop_rtp_mirror->callbacks.cls, raop_rtp_mirror->ntp, &h264_data);

                }    <----------------------------------------------------------added here after sending to gstreamer

                free(payload_decrypted); 
            } else if ((payload_type & 255) == 1) {
fduncanh commented 2 years ago

see the above comment (edited)

fduncanh commented 2 years ago

In short, do two different tests on modified uxplay;

(1) use a modified uxplay that prints the before- and after- decryption video packets with uxplay -d until "abgebrochen" happens. (use uxplay -d) This is in the slight hope that the "before" video packets are not encrypted (and just the audio is encrypted)

(2) use a different modified uxplay that dumps the bad video, to see if the audio might be properly decrypted and play correctly.

fduncanh commented 2 years ago

@arararar

I have installed Windows 10 in virtual box and installed free airmypc in it. I can connect from airmypc on the virtual Windows 10 PC to Uxplay running on the host Linux PC and I see exactly what you saw: the crash due to the assert, triggered by the failed decryption. Dont worry about the tests I asked you to do. I can do them much quicker, and I will post the results here of what I find

fduncanh commented 2 years ago

OK. The bad news is that both the sound and video are encrypted, and not being properly decrypted. I have a working test setup in that I can reproduced your issues. I doubt if I will find any simple fix, but will see if there is anything obvious. sorry.

fduncanh commented 2 years ago

I tried to see in airmypc could connect to shairplay, the "grandparent" of RPiPlay and audio-only player that uses older protocols, but no joy.

here is a collection of links to open-source airplay receivers. If any of them work with airmypc, one could port the decryption method to UxPlay

https://github.com/openairplay/open-airplay

fduncanh commented 2 years ago

more links https://awesomeopensource.com/projects/airplay

fduncanh commented 2 years ago

Closing this for now.

(In absence of ideas on how to find the encryption protocol used by airmypc)

will reopen if there are any developments

arararar commented 2 years ago

I wrote to the airmypc support, they seem to be willing to look into the matter:

Hi. As far as i understood: 1) Audio is mirrored OK, issue is only with video. 2) uxplay has issues decrypting the video stream.

Is that right?

Mirror screen ONLY again. Wait for error message or if you do not get error then wait for 1 minutes. Then Please Open AirMyPC Settings screen. Then hold the CTRL key and while it is down, click the support button in AirMyPC left menu. This should open AirMyPC debug folder. There you should see a file with the name debug.txt please send it to me.

Sincerely, Ben-AirMyPC Support On 11/30/2021 10:05 PM wrote:

Name Version : Newest

OS: Windows 10

Device Type and Version: Another

Question: Please help with information concerning this problem: https://github.com/FDH2/UxPlay/issues/25

If it works between airmypc and uxplay, we will order more licences for my collegues.

Yours, andrew

Try that way?

stackrainbow commented 2 years ago

I don't think audio works either, but I could be understanding the issue wrong. I cannot test audio as my uxplay server does not have audio.

fduncanh commented 2 years ago

yes

the issue is that both audio and video get incorrect final aesiv and aeskey.

The sequence of pair-setups and pair-verify POST requests is different between iPad (iOS 15.1) and airmypc clients. My first guess is that maybe ehcd_secret is miscalculated during the pairng process.

My second guess is that airmypc (which seem to report as a very old iOS version) may use an older protocol for processing

two forms of aes decryption are used: cbc for audio, ctr for video.

Both require a key and an "initial vector" (iv). there are four pieces of data sent by the client.

from these we need two keys and two iv's (one pair for audio one pair for video) ; an iv is 16 bytes.

1 .eiv (16 bytes)

  1. ekey (72 bytes)

  2. ecdh_secret (36 bytes, created during the pairing process, using "fairplay")

  3. streamConnectionID (I think 8 bytes)

eiv ekey and ecdh secret are processed to get the audio aes_cbc key and iv.

ekey ecdh_secret and streamConnection ID are processed to get the video aes_ctr key and iv.

Airmypc reportes sourceVersion 280.33 which I guess is a very old ios version.

My Ipad reports Source version 595.13.1 (iOS 15.1)

Its likely that some of the processing steps of the iv and key were simpler for old iOS, and added in later versions.

I will check some more for accounts of the older encryption scheme.

fduncanh commented 2 years ago

Correct UxPlay interaction with an iPad: Conforms to the clear explanation in https://github.com/SteeBono/airplayreceiver/wiki/AirPlay2-Protocol (This refers to a different implementation of an AirPlay server in C# I need to check in detail that it is exactly the same as the older one in UxPlay) (see section on "AirTunes - The Handshake")

Initialized server socket(s)
Accepted IPv4 client on socket 31
Local: 192.168.1.25
Remote: 192.168.1.136
Open connections: 1
httpd receiving on socket 31
conn_request
Handling request GET with URL /info
INFO len = -1484793884
httpd receiving on socket 31
conn_request
Handling request POST with URL /pair-setup
httpd receiving on socket 31
conn_request
Handling request POST with URL /pair-verify
httpd receiving on socket 31
conn_request
Handling request POST with URL /pair-verify
httpd receiving on socket 31
conn_request
Handling request POST with URL /fp-setup
httpd receiving on socket 31
conn_request
Handling request POST with URL /fp-setup
httpd receiving on socket 31
conn_request
Handling request SETUP with URL rtsp://192.168.1.25/7027027721913583545
DACP-ID: 41A43D016F643E8A
Active-Remote: 1542534805
Transport: null
SETUP 1
eiv_len = 16
ekey_len = 72
fairplay_decrypt ret = 0

at this point the decryption has been received from the client. Further processing occurs in lib/raop-buffer.c (audio) and lib/mirror-buffer.c (video)

Now compare to the different pairing sequence by UxPlay with airmypc There seems to be an extra pair-verify request before the pair-setup.

Handling request GET with URL /info
INFO len = -848185404
httpd receiving on socket 29
conn_request
Handling request POST with URL /pair-verify
httpd receiving on socket 29
conn_request
Handling request POST with URL /pair-setup
httpd receiving on socket 29
conn_request
Handling request POST with URL /pair-verify
httpd receiving on socket 29
conn_request
Handling request POST with URL /pair-verify
httpd receiving on socket 29
conn_request
Handling request POST with URL /fp-setup
httpd receiving on socket 29
conn_request
Handling request POST with URL /fp-setup
httpd receiving on socket 29
conn_request
Handling request SETUP with URL rtsp://10.10.100.4/6549601705745152332
DACP-ID: AF3AD2BCCCEC7317
Active-Remote: 1882964716
Transport: null

I will need to get more output to see if something is wrong here (maybe not)

fduncanh commented 2 years ago

I have verified that the pairing between airmypc and UxPlay (which produces the "ecdh_secret" used for decrypting communication between them) appear to be correct.

As part of this I found an ancient bug in RPiPlay (origin of UxPlay code) that silently skipped a step (in second call to pair-verify after pair-setup) where the client-server cryptographic connection is tested. I fixed this to make the test happen, and the connection with airmypc passes the test.

some quite intricate decryption of the cryptographic keys used to decrypt audio and video happen after that.

The original work that created the video part of RPiPlay (and hence uxplay) is described here. https://www.programmersought.com/article/2084789418/ ( see also https://github.com/SteeBono/airplayreceiver/wiki/AirPlay2-Protocol )

There is some comment in the first of these about signed versus unsigned integers causing decryption failures that seems interesting. Maybe airmypc does something slightly wrong (or deprecated) , but appleTV has something to detect this and fix it that we dont have? (a conversion between signed and unsigned?)

fduncanh commented 2 years ago

This is the interesting comment in https://www.programmersought.com/article/2084789418/

After the key and iv can be decrypted, the decrypted data is the H264 bare stream in avcc format.
At this point, the screen image data is parsed.

Tips: When the code is here, the streamConnectionID is converted to %lld format, which will cause 
the correct image to be displayed at the time of projection. All of them will be wrong data. Later, the 
og will find that the output streamConnectionID is negative, resulting in decryption error. In %llu format
fduncanh commented 2 years ago

@arararar

I've looked at the problem of airmypc, and am giving up unless new information emerges. AES decryption of both audio and video fails

The action takes place in lib/raop_handlers.h which is not a usual header file, but is just a device to keep the key parts of the code for the protocol in one place: it is included at the top of raop.c so it is really part of raop.c.

The pairing setup in raop_handlers.h line 171 pairsetup + line 198 pairverify (called twice) seems to have worked , as the second pairverify call seems to has succeeded with no errors. This has created a shared secret ecdh_secret for the client and server to use. The next two calls to fpsetup (line 257) set up the "fairplay" scheme that was originally used to protect Apples itunes music. Here it is used to decrypt a 72-byte "ekey" that is sent from the client to the server, to produce the 16 byte "aeskey" at line 374. This plus ecdh_secret are the two main ingredients of decryption together with "initial vectors" for audio (derived from eiv) and video (derived from streamConnectionID) . Since both audio and video decryption fail, I guess that the fairplay decryption step ekey -> aeskey is what has failed for airmypc. I have no way to test this. This could fail because airmypc uses a different (older?) protocol , but if it is doing that it must tell the server about it, and I didnt see this.

One difference between airmypc and iOS devices is that airmypc does the audio setup before the video one, and uses ALAC (ct=2) audio format instead of the AAC-ELD (ct = 8) format iPads use when mirroring.

Descriptions of the protocol can be found at https://github.com/SteeBono/airplayreceiver/wiki/AirPlay2-Protocol which is based on the original work which can be read here https://www.programmersought.com/article/2084789418/#4__1439

I doubt you will get any help with a decryption issue from the folks at airmypc, but who knows?

fduncanh commented 2 years ago

@arararar

I've fully documented your issue #25 as new issue #37.

(The uxplay -d debug output from UxPlay now shows all communication between client and server, and uxplay detects failure to decrypt both audo and video gracefully with an error message, not a crash).

If you can interest the AirMyPC people with this issue, refer them to issue #37.

Tell them that UxPlay is open-source, works perfectly with Apple clients, and can be built both on Linux and MacOS if they wish to test with AirMyPC.

If a clue for a solution is found, I would be happy to add an option for clients that use a modified protocol to UxPlay