search

facebookarchive / bistro

Bistro is a flexible distributed scheduler, a high-performance framework supporting multiple paradigms while retaining ease of configuration, management, and monitoring.
https://bistro.io
MIT License
1.04k stars 158 forks source link

Support Encrypted Traffic #4

Closed kevincox closed 5 years ago

kevincox commented 8 years ago

It might just be the docs being in a fresh state but I can't see anything about encrypting inter-node traffic. It would be nice to be able to run Bistro without trusting the network.

snarkmaster commented 7 years ago

Thanks for writing! Bistro uses facebook/fbthrift to communicate between the scheduler, which does support encrypting inter-node traffic. I'll do some research about how to set that up from scratch.

To start with, you would need some independent method to deploy some mechanism for mutual authentication & encryption on all hosts (e.g. X509 credentials). Once that's present, I think it should be fairly straightforward to tell fbthrift to use it.

There's more to this story, I'll comment again when I can speak with more certainty.

snarkmaster commented 5 years ago

@kevincox, sorry for the long wait, but I have something resembling an answer for you now.

Internally at FB, all Thrift traffic is automatically TLS-encrypted, see https://code.fb.com/security/service-encryption/

Of course, the specifics of how this configured are not at all useful for open-source users.

However, read on for a good hint.

Bistro's REST UI can be used over HTTPS as of this commit: https://github.com/facebook/bistro/commit/7d6945dcbb9ec8f84aae62c24d423e7bd677338a#diff-da35324a4141e60f4bb6b09fee92687eR106

This HTTPS setup uses the same security primitives as the TLS implementation of FBThrift.

So, this commit is an example of the fairly minimal configuration one needs to give to Thrift to make it encrypted.

I haven't actually tried to make it work, but it seems very doable.

If it's still relevant, let me know.

kevincox commented 5 years ago

I've decided against using Bistro but I imagine this would still be useful for various users.

snarkmaster commented 5 years ago

Thanks for the prompt response, @kevincox. I'm going to close this out until somebody explicitly asks for it, since inventing an authentication setup in a vacuum feels academic. Most of the work in setting up authentication is along these lines:

I'm not an expert in any of those, and would produce a worse presentation than materials already available on the internet.

So the only thing I can contribute is how to point Bistro at TLS certs & keys.