search

tkuester / taky

A simple python TAK server
MIT License
188 stars 43 forks source link

DPS: Enforce certificates #4

Closed tkuester closed 3 years ago

tkuester commented 3 years ago

Currently, the datapackage server will let unauthenticated clients connect. It should not.

tkuester commented 3 years ago

Well that's delightful.

Passed this to gunicorn:

options['cert_reqs'] = ssl.VerifyMode.CERT_REQUIRED

And when I try to do a datapackage search on ATAK 4.2.1.7 (b8b47239) I get an SSL error (TLSV1_ALERT_UNKNOWN_CA). It looks like this is a bug in ATAK.

tkuester commented 3 years ago

Installing the certificate on the Android device did not help, either.

doug-fitzmaurice-rowden commented 3 years ago

I think ATAK can be quite fussy about where CA certificates are installed, and the signing chain for certs.

Is the CA for the server the same as the CA for the client certificate? Are the CA(s) installed in ATAK's internal Trust Store database? Are there any intermediate certificates involved in the server or client chain?

I found that all CA and intermediate certificates had to be in the trust stores before client cert auth for HTTP calls would work.

doug-fitzmaurice-rowden commented 3 years ago

Another thought - have you tried packaging the .p12 client certificate file using Java's keytool? There are some extra settings that keytool puts in that seem to be important to Java applications, but are not widely supported otherwise.

Because I don't like using openssl or keytool I put together a little Go application to make p12s for ATAK, this should also add the correct settings to the trust store: https://github.com/rowdentech/go-p12

tkuester commented 3 years ago

Another thought - have you tried packaging the .p12 client certificate file using Java's keytool?

I haven't, but this is very helpful! This isn't a priority issue for me right now, but I'll give it a shot when I have some spare cycles!

doug-fitzmaurice-rowden commented 3 years ago

I've done some digging into this and something very strange is happening - a set of certificates that work happily for DPS transfer between ATAK-CIV and the OG server don't work between the same client and taky.

Testing using openssl s_client -connect showed that gunicorn was only returning the server cert rather than the full chain including the CA, but changing the server cert to be a chain didn't solve the problem.

OpenSSL and step certificate inspect show both servers returning identical certs, but ATAK talks to one and not the other. There must be something subtle about the way that gunicorn sets up the TLS connection that ATAK is very sensitive to.

The bit I find most confusing is that the error comes out of the boringssl C++ library, but if you try and connect to an OG server with an unknown CA the error comes from a Java library...

doug-fitzmaurice-rowden commented 3 years ago

Just found that I can replicate the issue with curl, taking ATAK out of the loop entirely:

$ curl --cert doug.crt --key doug.key --cacert ca.crt -k https://ogserver:8443/Marti/sync/search
{"resultCount":13,"results":...
$ curl --cert doug.crt --key doug.key --cacert ca.crt -k https://taky:8443/Marti/sync/search
curl: (77) schannel: next InitializeSecurityContext failed: SEC_E_UNTRUSTED_ROOT (0x80090325) - The certificate chain was issued by an authority that is not trusted.
tkuester commented 3 years ago

@doug-fitzmaurice-rowden -- I just tested out your pull request. Wow, I can't believe I'd done something so stupid. Thank you for working so hard to find my mistake!

I tested it out the other day, it looks like this is ready for a merge. So far I've checked...

  1. That I can upload, search, and download data packages and video endpoints on the server
  2. That the client polls and correctly reports the server version
  3. That when using a regular browser without the client certificates, I get an SSL error

This is an extremely welcome pull request! May I credit you in a README or some other document?

There is one last thing I'd like to do before closing this issue. Now that I can check client certificates, this allows me to truly do public/private data packages. If your client ID does not match the user's callsign, you may not download their private data packages. That should be a quick fix, but I'm not able to get it in right now.

doug-fitzmaurice-rowden commented 3 years ago

No worries! It wasn't obvious at all from the Gunicorn docs, and the SSL protocol error messages are also unhelpful in not telling you which end of the connection had the CA issue.

Sure, happy to have a mention in the readme, thanks! I'll have a go at the private data package fix now as well..

tkuester commented 3 years ago

That would be welcome help!

There is a field known as "Submission User" that is documented here, but I'm not particularly sure if ATAK as a client does anything with it yet.

Needless to say, I think it would be simple enough to copy the SSL certificate's client field on submit and log that for "Submission User".

My guess is that the safest thing to do is assume user uploads should be marked as private (and therefore should not show up in search results). However, since the client has no way to request that a datapackage should be public, that would be extremely limiting for users with less security concerns.

Down the road it would be cool to implement a user friendly Web UI that would allow easier management of datapackages... but I think what would be good is to add a config item for the DPS -- "mark all user uploads as public", that is default off. (Or perhaps, for usability's sake, it should be default on? What are your thoughts?)