ios9 mirror 【*** ERROR decryption of video packet failed 】

[hushilin]

ios9 mirror error。 ios10 OK

[fduncanh]

This is a new feature in the code, which I may have to change. Please post exactly which version of UxPlay you are using, and the debug output of uxplay -d. (up to the ERROR)

Do both audio and video fail?

I need to know what ios9 reports as its "sourceVersion" this is seen in the debug output

[hushilin]

控制台部分日志：

raop_rtp_mirror accepting client
raop_rtp_mirror width_source = 304.000000 height_source = 540.000000 width = 318.000000 height = 564.000000
raop_rtp_mirror sps size = 20
raop_rtp_mirror pps size = 4
raop_rtp_mirror video ntp = 1639552615214109, now = 1639552615159764, latency = -54345
*** ERROR decryption of video packet failed 

define VERSION "1.44"

@fduncanh

[fduncanh]

NO its not that.

Is this a complete failure or just a single event.

What happens when you run uxplay -d with ios 9

[fduncanh]

run uxplay -d

then uxplay -d > filename 2>&1

to send debug output to the file ``filename```

It will tell you if all video ((and audiio) fails decryption. or just one frame.

[hushilin]

@fduncanh erbao:Debug admin$ ./uxplay -d using system MAC address ac:de:48:00:11:22 Initialized server socket(s) raop port = 51458 Accepted IPv4 client on socket 28 Local: 192.168.0.199 Remote: 192.168.0.112 Open connections: 1 client sourceVersion 260.26 (no hash of aeskey if <= 280.33) raop_rtp_mirror starting mirroring Accepted IPv4 client on socket 32 Local: 192.168.0.199 Remote: 192.168.0.112 Open connections: 2 *** ERROR decryption of video packet failed

[fduncanh]

Yes I see. (but some how the -d option is not giving the full debug output for you)

ios9 sourceVersion 260.26 is the issue. I will fix the new code right now.

[fduncanh]

Thank you for reporting this. Its a change I just made yesterday.

[hushilin]

未命名.txt

完整日志文件

[hushilin]

Is this a problem with the video renderer, or an airplay protocol error?

[fduncanh]

No it is a special protocol change to accomodate a Windows Client AirMyPC that uses an older protocol. see issue #37

This works with a true Apple TV and declares sourceVersion 280.33.

I had assumed that the AppleTV recognizes 280.33 as too old for the modern protocol, and uses the old one for clients with 280.33 or earlier. Clearly I was wrong since it seems iOS9 (250.26) uses the new protocol I will detect AirMyPC a different way, and use the old protocol only for AirMyPc.

I guess iOS10 has a sourceVersion greater than 280.33

[hushilin]

IOS10 : client sourceVersion 320.20 (no hash of aeskey if <= 280.33) IOS14: client sourceVersion 540.31 (no hash of aeskey if <= 280.33)

我的两个设备连接 日志

[fduncanh]

Your debug output confirms what guessed.

Thank you

UxPlay 1.43 should work for you (see releases) till I fix 1.44 (not yet a release)

[fduncanh]

I dont know what AppleTV is using to know that AirMyPc uses the old protocol if it is not using sourceVersion....

Please check that UxPlay 1.43 works with ios9

[hushilin]

You are so hard！ Which country are you from？

[fduncanh]

planet earth

[hushilin]

nice!。 1.4.3 支持 iOS 9

[hushilin]

The mirroring connection is successful, why is there no sound?

[fduncanh]

No sound with 1.43 and ios9

was there sound but no video with 1.44?

[fduncanh]

In modern iOS AESkey is hashed with ecdh_secret

It is then used for both sound and (in a complicated way) for video. AirMyPC uses unhashed AESkey for both. its possible ios9 used hashed for video and unhashed for sound (?)

both are hashed in 1.43 none are hashed in 1.44 that you have, for sourceVersion < 280.33 .

[hushilin]

1.43 和 ios9 没有声音

1.44 有声音但没有视频吗？

1.43 and 1.44 version - iOS9, iOS10 都没有声音 Error ： *** ERROR decryption of audio (compression_type 8) failed

[hushilin]

No sound is just an GStreamer audio renderer problem, Isn't it the problem in the airplaylib protocol code?

Because I only need the code in airplaylib to parse with ffmpeg

[fduncanh]

If you get that ERROR message for audio, it means that the AES decryption of the audio stream failed. if you get the ERROR for video. it means that decryption of the video stream failed.

(These are two separate decryption processes).
The Error is detected by examining the first byte of eack packet in the audio or video stream just before giving it to the GStreamer renderer, but happens earlier in the decrypt process (raop_buffer.c (audio), mirror_buffer.c (video) in lib)

0. client and server create a "shared" secret ecdh_secret
1. (client) sends ekey (72 bytes) and eiv (16 bytes) to server
2. server uses reverse-engineered FairPlair decryption (playfair) to extract (16 byte ) aeskey from ekey
3. aeskey + ecdh_secret -> sha512 hash -> (decrypted) aeskey

(aeskey, eiv) are the (key,iv) pair for audio AES-CBC decryption AES CTR video decryption uses a (key, iv) pair derived from aeskey and streamConnectionID.

old protocol (seen in Windows Airplay-client emulator AirMyPc) : omit step 3. Airplay 1 audio (audio only) omits step 3. really old ios (5 maybe) did not encrypt screen mirror video.

1.43 and 1.44 should behave identically for iOS 10 and later. do audio and video both work for you in ios14?

which iOS versions do you think should be supported?

I am supporting the old protocol for AirMyPC because it is currently used and a teacher wants if for a class where some students have iPads, other have some Windows surface which can run AirMyPC.

[fduncanh]

The failure of video decryption is detected in raop_rtp_mirror.c (look for "nalu") and is marked by setting the first byte of the video frame to 0x1.

For audio (raop_rtp.c) the code where the decryption takes place doesnt know it the audio is ALAC or AAC (but could test for both, I suppose) . Because of this, testing for bad decryption was postponed till gstreamer, which has been told what audio format to expect.

bad decryption was never seen before a user tried to make uxplay work with the AirMyPC client. its only detected since 1.43

Maybe no-one reported failures with old iOS versions.

[fduncanh]

@hushilin please test with latest version of 1.44

You can try to get audio on ios9 ios10 to decrypt properly by adding their User-Agent strings in lib/global.h

replace OLD_PROTOCOL_AUDIO_CLIENT_LIST "AirMyPC/2.0;xxxx" by OLD_PROTOCOL_AUDIO_CLIENT_LIST "AirMyPc/2.0;AirPlay/260.26;AirPlay/320.20;xxxx"

for both iOS 9 and iOS10 clients you are testing.

[hushilin]

@hushilin please test with latest version of 1.44

You can try to get audio on ios9 ios10 to decrypt properly by adding their User-Agent strings in lib/global.h

replace OLD_PROTOCOL_AUDIO_CLIENT_LIST "AirMyPC/2.0;xxxx" by OLD_PROTOCOL_AUDIO_CLIENT_LIST "AirMyPc/2.0;AirPlay/260.26;AirPlay/320.20;xxxx"

for both iOS 9 and iOS10 clients you are testing.

You modified version 1.4.4. I tested ios9 and 10 and there is still no sound.

1.44 version iOS9 video mrrir display OK👌

[fduncanh]

Please try with a new version of 1.44 I just committed

Thank you

[hushilin]

Just modify lib/global.h OLD_PROTOCOL_AUDIO_CLIENT_LIST "AirMyPc/2.0;AirPlay/260.26;AirPlay/320.20;xxxx" ？？？

After I modified it myself, compiled it I tested ios9 and 10 and there is still no sound.

[fduncanh]

This probably means that UxPlay and RPiPlay have never supported ios9 and ios10 audio...

Please test with http://github.com/FD-/RPiPlay to make sure that it is not something I have broken.

I will see what I can learn about the history of these iOS versions

[fduncanh]

I dont' see any reference to anything older than iOS 12.4 in the history on the RPiPlay site, so my guess is RPiPlay won't work with iOS 9 and 10. (But perhaps only some small change in protocol might be need to be guessed)

[hushilin]

Want to ask you void video_process (void *cls, raop_ntp_t *ntp, h264_decode_struct *data) in h264_decode_struct is the data returned in standard H264 format ？

I want to use the iOSVideoToolbox.h library CMVideoFormatDescriptionCreateFromH264ParameterSets to parse the data, but it fails

[fduncanh]

Did you test RPiPlay?

The history is shairplay -> AirPlayServer ( by dsafa22 , in java for Android, but contains C from modified shairplay) -> RPiPlay -> UxPlay

For AirServer details see dsafa22's discussion paper https://github.com/FDH2/UxPlay/wiki/AirPlay2

he explored with iOS 12.0.1 User-Agent: AirPlay/371.4.7 so wont help for iOs 9 and 10.

However the fact the video is working makes it possible that the audio setup is simpler that the one in RPiPlay, but more complex than that in shairplay. The audio in the AirMyPC app is the same as in shairplay.

[fduncanh]

Video data is not quite pure h264. see what happens after the decryption in raop_rpt_mirror.c (look for nalu)

The decrypted data is converted back to pure h264 there. See dsafa22's paper.

[fduncanh]

void video_process (void cls, raop_ntp_t ntp, h264_decode_struct *data)

The data here has already been converted to pure h264.

[hushilin]

When I first call void video_process (void *cls, raop_ntp_t *ntp, h264_decode_struct *data) data.length == 30 I want to getnalu sps pps

I found an example elsewhere. The data returned by first call, data.length is 36 or 37, can be decoded successfully with VideoToolbox.h

That’s why I asked you whether the returned standard 264 data

[hushilin]

Ios AirplayMirror he did not provide the source code

void airplay_data_receive(unsigned char buffer, long buflen, int payload,void ref) data.length is 36 or 37, can be decoded successfully with VideoToolbox.h

i'm not very familiar with C language code, can you help me see it?

[hushilin]

libairplaymirror.a The static library has serious bugs ，Uncertain collapse

[fduncanh]

good find ! Its written in Objective C (Apple language) not C though.

maybe the audio decryption protocol for iOS 9,10 11 is in there somewhere I now think UxPlay as written is for ios 12 and later note that (from RPiPlay notes)

For iOS 9, Apple made considerable changes to the AirPlay protocol in 2015, including audio and mirroring. 
Apparently, the audio protocol was only slightly modified, and a minor change restored compatibility. 
For mirroring, an additional pairing phase was added to the connection establishment procedure, consisting 
of pair-setup and pair-verify calls. Seemingly, these were added in order to simplify usage with devices that 
are connected frequently. Pair-setup is used only the first time an iOS device connects to an AirPlay receiver.
 The generated cryptographic binding can be used for pair-verify in later sessions. Additionally, the 
stream / stream.xml endpoint was replaced with the info endpoint (only available as binary plist AFAICT). 
As of iOS 12, the protocol introduced with iOS 9 was still supported with only slight modifications, albeit as 
a legacy mode. While iOS 9 used two SETUP calls (one for general connection and mirroring video, and 
one for audio), iOS 12 legacy mode uses 3 SETUP calls (one for general connection (timing and events),
one for mirroring video, one for audio).

.

[fduncanh]

I'll look some more at your uxplay -d output for ios9 can you post the same for ios10?

[hushilin]

The internal implementation of libairplaymirror.a should be implemented in C language. Object-C just calls it. But the source code of libairplaymirror.a is not provided

I want you to help me reason about why it calls data.length for the first time is 36.

uxplay void video_process (void *cls, raop_ntp_t *ntp, h264_decode_struct *data)data.length == 30

[fduncanh]

} else if ((payload_type & 255) == 1) { // The information in the payload contains an SPS and a PPS NAL line 355 raop_rtp_mirror.c

[fduncanh]

            // The sps_pps is not encrypted

[fduncanh]

     // Copy the sps and pps into a buffer to hand to the decoder

[fduncanh]

If you post the output (with working video, but broken audio of newest uxplay -d

for both ios9 and ios10 I might now see why audio didnt work,

[hushilin]

thank you very much The ios hard decoding succeeded.

[hushilin]

如果您发布输出（伴随工作视频，但最新的 uxplay -d 的音频已损坏）

对于 ios9 和 ios10， 我现在可能已经明白了原因了，

do you still need uxplay iOS10 running log information?

[hushilin]

there is no sound in iOS9,10 is there no way to solve this problem?

sorry, my English is not very good, maybe the tone of the question is very strange

please forgive me

[fduncanh]

Please post the full debug logs for both iOS9 and 10. (with working video)

Maybe I can see what goes wrong with sound (but maybe not)

The current support is ios12 or later.

[hushilin]

You haven't slept yet?

This is the debug log： ios9_log.txt ios10_log.txt

[fduncanh]

Interesting. These debug logs are using the "old protocol for audio (the unhashed AES key for audio) and most of the audio frames seem to be correctly decrypted, just some fail. I see 8 good frames then 1 bad one in the debug.

aop_rtp audio: ntp = 1639665663627271, now = 1639665663382430, latency=-244841, rtp=3754258338
raop_rtp audio: ntp = 1639665663638156, now = 1639665663382482, latency=-255674, rtp=3754258818
raop_rtp audio: ntp = 1639665663649040, now = 1639665663382491, latency=-266549, rtp=3754259298
raop_rtp audio: ntp = 1639665663659924, now = 1639665663382499, latency=-277425, rtp=3754259778
raop_rtp audio: ntp = 1639665663670809, now = 1639665663382506, latency=-288303, rtp=3754260258
raop_rtp audio: ntp = 1639665663681693, now = 1639665663382556, latency=-299137, rtp=3754260738
raop_rtp audio: ntp = 1639665663692577, now = 1639665663382566, latency=-310011, rtp=3754261218
*** ERROR decryption of audio frame (compression_type 8) failed 
raop_rtp audio: ntp = 1639665663703462, now = 1639665663382604, latency=-320858, rtp=3754261698
*** ERROR decryption of audio frame (compression_type 8) failed 
raop_rtp_mirror video ntp = 1639665663473797, now = 1639665663404596, latency = -69201
raop_rtp audio: ntp = 1639665663649040, now = 1639665663412494, latency=-236546, rtp=3754259298
raop_rtp audio: ntp = 1639665663659924, now = 1639665663412569, latency=-247355, rtp=3754259778
raop_rtp audio: ntp = 1639665663670809, now = 1639665663412669, latency=-258140, rtp=3754260258
raop_rtp audio: ntp = 1639665663681693, now = 1639665663412696, latency=-268997, rtp=3754260738
raop_rtp audio: ntp = 1639665663692577, now = 1639665663412712, latency=-279865, rtp=3754261218
raop_rtp audio: ntp = 1639665663703462, now = 1639665663412728, latency=-290734, rtp=3754261698
raop_rtp audio: ntp = 1639665663714346, now = 1639665663412743, latency=-301603, rtp=3754262178
*** ERROR decryption of audio frame (compression_type 8) failed 

I ask why no audio was heard, since most audio frames seem to be good. Maybe the volume is turned off? There is some negotiation about volume between client and server.

---

I now removed the "old protocol" for ios9/10 from lib/global.h

can you repeat the test again with a new download of uxplay (or just edit global.h so it says

#define OLD_PROTOCOL_AUDIO_CLIENT_LIST "AirMyPC/2.0;xxx"
#define OLD_PROTOCOL_VIDEO_CLIENT_LIST "AirMyPC/2.0;xxx"

Or try with 1.42.

I would then expect to see EVERY audio frame fail. does that happen?

[hushilin]

new ios10_log.txt

@fduncanh In addition, I would like to ask, is the video projection data played on the web page different from the screen mirroring data?

Screen mirroring can display, video projection screen displays black