scintill / vince

VNC viewer for Roku OS (GPLv3+)
Other
3 stars 2 forks source link

UI issues #4

Open scintill opened 4 years ago

scintill commented 4 years ago
akohlsmith commented 1 year ago

Please don't take this as some rando trying to "manage" you - just seemed a handy place to add feature requests...

akohlsmith commented 1 year ago

I managed to get this working (in a VERY hackish way) with OSX (type 30) auth; I wrote a very stupid/simple python server which I had Vince give the auth challenge to, it computes the response (with a fixed user/pass) and sends it back. This allows Vince to start displaying the screen, but there are several odd rendering issues; the scaling doesn't work (it looks like Vince only scales up, not down, so > 1920x1080 spits out errors about the received rectangle being too big for the UI), there are gaps between each received block of 100 pixel high rectangles, and it's terribly slow, but it does work. There'd be a LOT of work involved in trying to get something like TRLE or ZRLE implemented, and I'm pretty sure BrightScript is just not capable of doing the bignum math involved in the Diffie-Hellman Key Acceptance. This is quite cool from a proof of concept though.

The auth mechanism is documented here, but in a nutshell:

  1. receive 2 byte generator, 2 byte key length, then a key length number of bytes for both the modulus and remote's public key.
  2. generate a key pair using the provided generator and key length.
  3. generate a shared secret using the DH Key Agreement algorithm (explained below)
  4. generate the MD5 hash of the shared secret; this will be a 16-byte value and used as the AES encryption key
  5. create two 64-byte buffers: put the zero-terminated username in the first and the zero-terminated password in the second. Either fill the empty space with zeroes or (more secure) random characters, but the user/pass must be zero-terminated.
  6. encrypt the 128 byte buffer with AES in ECB mode with no additional padding. The 16-byte MD5 hash from 4) is the encryption key.
  7. Send the encrypted response from 6) along with the public key from 2) back to the server.

See RFC 6143 for the details on the other bits of the protocol.

DH Key Agreement is actually more straightforward than I thought (I am not a crypto guy). Explained for people like me here. In python, this is what I did (thank you python for handling bignums as regular old ints!). My hacked up Vince is sending length of key, generator, modulus and pubkey as base64-encoded JSON strings, so the first four lines are just undoing that. My VNC server is always using 128 bit modulus/keys, so n is 128 for me:

    n = data["len"]
    gen = int.from_bytes(base64.b64decode(data["gen"]), byteorder="big")
    mod = int.from_bytes(base64.b64decode(data["modulus"]), byteorder="big")
    their_key = int.from_bytes(base64.b64decode(data["pubkey"]), byteorder="big")

    # my_privkey, my_pubkey - (big) integers of the generated public and private key
    # shared - (big) integer of the shared secret
    # sh_hash - MD5 hash of the shared secret, used as encryption key for 128-bit AES in ECB mode
    my_privkey = int(binascii.hexlify(os.urandom(16)), base=16)
    my_pubkey = pow(gen, my_privkey, mod)
    shared = pow(their_key, my_privkey, mod)
    sh_hash = hashlib.md5(shared.to_bytes((shared.bit_length() + 7) // 8))

    # u/p - username/password strings
    # ua/pa - username/password byte arrays (zero-filled)
    # upa - combined user/pass array. 128 bytes long.
    u = "vnc_username_here"
    p = "vnc_password_here"
    ua = bytes(u, encoding='utf-8')
    ua += b'\0' * (64 - len(ua))
    pa = bytes(p, encoding='utf-8')
    pa += b'\0' * (64 - len(pa))
    upa = ua + pa

    # rspa - encrypted 128-byte user/pass array
    # rsp - base64 encoded encrypted blob and public key in JSON format
    cipher = AES.new(sh_hash.digest(), AES.MODE_ECB)
    rspa = cipher.encrypt(upa)
    rsp = "{"
    rsp = rsp + "\"response\": \"" + base64.b64encode(rspa).decode('utf-8') + "\""
    rsp = rsp + ", \"pubkey\": \"" + base64.b64encode(my_pubkey.to_bytes((my_pubkey.bit_length() + 7) // 8, byteorder="big")).decode('utf-8') + "\""
    rsp = rsp + "}"