pair-setup-pin and modernising Airplay v2 protocol?

fduncanh commented 8 months ago

EDIT see progress on this at https://github.com/FDH2/UxPlay/tree/pair-pin2

pair-pin setup now works (use the -pin option)! (2023-11-24)

There is a (C-implemented) library that seems to have non-legacy pairing support:

https://github.com/ejurgensen/pair_ap

**EDIT libsodium is used partially in pair_ap, but there is no need to use anything other than OpenSSL for UxPlay

this uses libsodium, and since the modern AirPlay protocol is bases on an initial handshake with SRP6a (which is deprecated in OpenSSL3) its not obvious how to implement it using libcrypto fromOpenSSL3.
~~libsodium is available in linux *BSD, windows(MSYS2), macOS(homebrew, MacPorts), and can be natively compiled~~

Does anyone have any thoughts on this? (@thiccaxe ?)

thiccaxe commented 8 months ago

I don't see a glaring issue with libsodium. It seems to have windows support as well.

However, the code you linked seems to require openssl or gcrypt. Not sure if this is used because I did not see any usage in the files.

if it is possible drop openssl altogether and use sodium, that might be nice.

fduncanh commented 8 months ago

~~Too bad, it might need adding a new dependency on libsodium in addition to OpenSSL then.~~

~~nce the libsodium use involves calls crypto_*, these are the needed uses of libsodium, it seems:~~

./pair_fruit.c:  crypto_sign_keypair(sctx->public_key, sctx->private_key);
./pair_fruit.c:  crypto_sign_ed25519_sk_to_pk(vctx->client_public_key, vctx->client_private_key);
./pair_fruit.c:  ret = crypto_scalarmult(vctx->client_eph_public_key, vctx->client_eph_private_key, basepoint);
./pair_fruit.c:  uint8_t shared_secret[crypto_scalarmult_BYTES];
./pair_fruit.c:  uint8_t signature[crypto_sign_BYTES];
./pair_fruit.c:  crypto_sign_detached(signature, NULL, data, *len, vctx->client_private_key);
./pair_fruit.c:  ret = crypto_scalarmult(shared_secret, vctx->client_eph_private_key, vctx->server_eph_public_key);

thiccaxe commented 8 months ago

Looks like libsodium does not implement the sha hash algorithms.

It seems libgcrypt/openssl are used only for the sha hash and "bignum" operations. I guess you could use https://mattmccutchen.net/bigint/ or something, but at the end of the day, I doubt it is worth it?

thiccaxe commented 8 months ago

Looks like the purpose of libsodium is to "hide away" the details of the encryption schemes used and just present a uniform/opinionated interface for high-level crypto operations such as signing, encryption, key exchange, etc. Furthermore, libsodium places a high emphasis on security, which is not too relevant for this project.

We can't replace the relatively low-level crypto in airplay protocol with libsodium. So I think the approach that pair_ap took is a good choice here. If working in gcrypt is possible, that might make sense.

Another long-shot approach is to create a separate library - call it fruitcrypto or something - which can deal with the choices of sodium/openssl/gcrypt. This'll also take the crypto stuff out of uxplay and allow this project to focus on the gstreamer/x/wayland/session details. Just a thought.

fduncanh commented 8 months ago

I'm not sure how different modern pair_verify etc is from what we do now. It's good to know that that the protocol is better understood now, in case Apple stops supporting the legacy protocol. I did an exploratory attempt to implement pair-setup-pin, based on this client-side description,

https://htmlpreview.github.io/?https://github.com/philippe44/RAOP-Player/blob/master/doc/auth_protocol.html

https://github.com/philippe44/libraop

and got as far as sending the current 32-byte PK (instead of a 256 byte SRP6 one) and getting a 256-byte client pk + "proof" back from the client (but probably broken)

Most of this other work seems to be for audio, but its the same crypto, it seems.

fduncanh commented 8 months ago

Here is a version of SRP6 (or a precursor) that seems to only only use "bignum" from openssl, that pair_ap derives from, (see pair_fruit.c)

so maybe libsodium is not in fact needed.

https://github.com/cocagne/csrp

thiccaxe commented 8 months ago

https://github.com/postlund/pyatv/blob/master/pyatv/auth/hap_srp.py

Here is the pyatv implementation

thiccaxe commented 8 months ago

Pyatv depends on https://github.com/idlesign/srptools which doesn't seem to depend on anything special.

Pyatv uses 25519 and HKDF.

thiccaxe commented 8 months ago

Another long-shot approach is to create a separate library

Any thoughts on this? Something like pyatv but for the server, not a client. and in cpp

fduncanh commented 8 months ago

OpensSLL has a lib_srp.c that phillipe44's libraop usesin his pairing.cpp.

But ... openSSL has deprecated it (without replacement, so it's under sentence of death). Still, looking at the openSSL code, its all done basically with BIGNUM calls, and only the "2048" bit (256 byte)group is being used by Apple, it seems. It seems to me that it can be all done with openssl(without lib_srp, just reproducing the specific parts used in libraop with bignum).

WE dont exactly know what the server does (lib_srp shows some differences between server and client, but we dont know what changes apple made, since they modified srp slightly), but maybe it doesnt matter, we just need to supply proofs to the client who doesnt get to know the servers private key anyway.

I am wondering about the relation between pair_verify in uxplay and what e.g. libraop or pair_ap do.

I am testing with the bits for legacy pairing and require password switched on, so my ios clients ask for pin-pair-start etc. There is a uxplay branch "pair-pin" which has made a start, and displays a pin but not much more yet. I was confused by the 256 byte pk that is part of srp "2048", but now I understand more.

for now the new code is in c not cpp.

thiccaxe commented 8 months ago

when I use the pair-pin branch, the uxplay server IS getting discovered by mdns, but it is NOT showing up as a airplay server in either screen mirroring or audio, UNLESS the -pair flag is passed¸ is this intended behavior? (fixed by #230 )

fduncanh commented 8 months ago

for the moment. The -pair option restores the byte for legacy pairing and adds the byte for "require password" (maybe, I dont recall)

I am also not sending the correct 256 bytes PK. I will fix that soon, as my understanding develops. Something got broken, as well: its supposed to act normally without the -pair option.

This thing is just an exploration so so far.

fduncanh commented 8 months ago

Given that openssl has deprecated lib_srp, and will remove it at some point, the easiest way to include SRP might be to import the reference implementation

https://github.com/cocagne/csrp/tree/rfc5054_compat

(with mods taking account of the tweaks in hashing added by apple, mentioned in both pair_ap and libraop).

https://htmlpreview.github.io/?https://github.com/philippe44/RAOP-Player/blob/master/doc/auth_protocol.html

The Uxplay WIKI now has a page with collected links to various protocol implementations

fduncanh commented 8 months ago

The first 400 lines of pair_ap/pair_fruit.c is essentially just the reference srp implementation (MIT License) by Tom Cocagne, modified to allow the option of using gcrypt instead of openssl.

So its easy to find the changes made because of Apple's mods to the standard srp

fduncanh commented 8 months ago

Update: pair_app/pair_fruit.c contains only the client part of the reference implementation, which also has the server part too. The pair_app author removed the server parts.

thiccaxe commented 8 months ago

The pair_app author removed the server parts.

This agrees with my findings - the example server can't deal when using "fruit" pairing on the example-client. homekit and transient work.

fduncanh commented 8 months ago

closing, as this is not really an issue, but work is ongoing in UxPlay branch pair-pin, and will probably eventually succeed depending on available time.

https://github.com/FDH2/UxPlay/tree/pair-pin

Use links are available in the Wiki https://github.com/FDH2/UxPlay/wiki/AirPlay2-protocol

fduncanh commented 7 months ago

Progress report:

with guidance from http://github.com/philippe44/RAOP-player, http://github.com/ejurgensen/pair_ap and http://github.com/cocagne/csrp , the initial Apple-modified SRP6a (Secure Remote Password protocol) part of the pair-pin protocol is implemented and working, to the point that after successful use of the 4-digit pin, client and server have a "shared secret" K, the SRP6a "session key".

See the pair-pin branch of UxPlay https://github.com/FDH2/UxPlay/tree/pair-pin .

fduncanh commented 7 months ago

Progress in the pair-pin branch https://github.com/FDH2/UxPlay/tree/pair-pin is now documented in its README. Help welcome!

fduncanh commented 7 months ago

~~hit a "roadblock" in implementing the last part of pair-pin-setup.~~

Seems that a true apple TV server modifies the "nonce" (the client nonce is "add 0x01 to byte 15 of iv") for encrypting (using AES GCM 128) its public key which it sends to the client so the client can verify the shared secret as the final step of pair-pin-setup

(open source clients just don't care about verifying the server's epk, true Apple ones do, it seems).

AFAIK no-one has found this detail for "Legacy pairing". HomeKit-based pairing (as in pair_ap) seems better understood.

EDIT: ~~tried all 16 x 256 cases where the server's iv differs from the client iv by a single byte: no success.~~

EDIT2: found it! the server nonce is just to again "add 0x01 to byte 15 of iv" (in addition to the 0x01 initially added to decrypt the client epk). (there was a bug in my initial AES GCM 128 decryption, so I didn't initially find the nonce)

The "test all possibilities" method found the nonce after fixing the decryptor!

fduncanh commented 7 months ago

A commercial implementation of a server for AirPlay uses similar features to UxPlay, except for AES-GCM-128 encryption on "control channel" which we dont have (its added in the experimental code, as part of pair-pin-setup step 3)

AirPlay mirroring streams are encoded using H.264 video compression and transmitted as TCP packets. Resolution can be up to 1080p at 60 fps and Apple recommends a bandwidth of 25 Mbps per mirroring stream. Average bitrates will be significantly lower, depending on the content.

The latency for AirPlay on AS** is around 150 ms under good network conditions.

AS** implements security features of AirPlay. Encryption and key exchange method vary slightly based on the client (iOS) version. Starting with iOS 9 and newer, the keys are exchanged using industry standard ECDH key exchange (Curve25519). The on-screen PIN can be used to authenticate and guard against MIT spoofing and is implemented in terms of SRP (Secure Remote Password protocol). AirServer encrypts all privacy relevant content whenever possible. Our AirPlay audio implementation uses AES-128-CTR over UDP, control channel uses AES-GCM-128, and AirPlay video uses AES-128-CBC over secondary TCP channel.

thiccaxe commented 7 months ago

Amazing work! It seems like the client expects the server to retain pairing keys, so when you re-connect the client while using the same uxplay instance (or rather, the same keypairs), things fall apart a bit.

thiccaxe commented 7 months ago

Accepted IPv4 client on socket 34
Local: 192.168.1.35
Remote: 192.168.1.32
client requested pair-setup-pin, datalen = 86
request header: Content-Length: 86
Content-Type: application/x-apple-binary-plist
CSeq: 1
DACP-ID: -
Active-Remote: -
User-Agent: AirPlay/745.8.1

pair-setup-pin:  device_id = -
client requested pair-setup-pin, datalen = 347
request header: Content-Length: 347
Content-Type: application/x-apple-binary-plist
CSeq: 2
DACP-ID: -
Active-Remote: -
User-Agent: AirPlay/745.8.1

*** ERROR: Client Authentication Failure (client proof not validated)
Connection closed for socket 34
^CStopping...
warning: queue 0x- destroyed while proxies still attached:
  wl_subcompositor@7 still attached
  wl_seat@6 still attached
  xdg_wm_base@5 still attached
  wl_compositor@4 still attached
  wl_registry@2 still attached

fduncanh commented 7 months ago

Yes, I havent figured out how that should be done yet. (or how the previously-authenticated client indicates it expects to skip the pin)

I turned out that we already had the pair-verify already stuff done in existing code. Only the SRP (Secure Remote Password) pair-pin-setup was missing.

The server protocol was mostly guessable from the client implementations, there was just one missing undocumented puzzle involving a "nonce" I needed to solve, and the pair-ap example-client interacting with a real apple TV 3rd gen was key in finding it. (an extra iv[15]++ applied to the AES GCM 128 key + iv pair, after decrypting the client epk, before encrypting the server epk send as a return message. None of the open source client implementations check the server epk (encrypted public key) but the real Apple clients do, and refuse to pair if the server epk is invalid.

I assume that how to handle the saving of the SRP user data and handling the reconnect will become clear eventually.

One mystery: I dont understand what needs to happen for full encryption of server client communication after pairing is validated.

fduncanh commented 7 months ago

Issue is fixed (by a minor change, commenting out a test).

Right now the server doesn't care about remembering who has previously authenticated. Just the Apple clients do, it seems.

fduncanh commented 7 months ago

@thiccaxe The pair-setup-pin implementation seems to be fully working now. It would be great it you tested it.! Ready for release. (The pair-setup without pin works as before).

fduncanh commented 7 months ago

@thiccaxe It (pair-setup-pin) now seems to be working perfectly., and is merged into the master branch (for release as 1.67 in a few days)

FDH2 / UxPlay

pair-setup-pin and modernising Airplay v2 protocol? #228