Vault versus Vormetric / SafeNet / Porticor

Miserlou commented 9 years ago

Hey guys!

Amazing project - the world has needed a Free Software package like this for a long time.

I think I understand where you're going with this, but on your 'versus' page, you don't list any of your direct commercial competitors, namely Porticor, SafeNet's Virtual KeySecure, and Vormetric's Transparent Encryption for AWS. You seem to be feature-competative - but how do you really stack up against the non-Free options out there? Are you HIPAA compliant, for instance? Is Vault truly production ready?

I'd absolutely love to use Vault over these extremely expensive commercial offerings if we could (even if you guys put Vault up as a paid AMI on AWS, I'd be willing to pay for it) - but I want to you guys to come out and say that you're ready to compete with them first.

<3! R

armon commented 9 years ago

Hey @Miserlou! The lack of documentation is largely because we don't know much about these systems. Commercial systems tend to say very little about actual technical details and provide a lot of marketing instead. This makes it hard to provide a meaningful comparison.

I'd be happy to review a PR detailing the comparison or to look over a specific product if you'd like. It's hard for us to do for every product however, at some point it's an exercise left to reader.

Miserlou commented 9 years ago

Thanks for the swift reply, Armon!

Since Porticor was recently purchased by Intuit and is no longer accepting customers, I think the two that are really worth mentioning are SafeNet's Virtual KeySecure and Vormetric's Transparent Encryption.

SafeNet's Virtual KeySecure "centralizes cryptographic processing, security policy and key management – all in a FIPS-validated hardened virtual security appliance".

The big claims here seem to be: many key types, AD/LDAP integration, signed logging, audit trails and secure back ups, time and number based access restriction in addition to user policy restriction, plus their own "format preserving encryption" (which is kind of no big deal in my opinion). Though they claim a "max concurrent clients" of 100, this is actually a licensed-based value, and concurrent clients cost about ~$9,000 each, in addition to the $12,000 upfront cost for the appliance. Does Vault have a "max concurrent clients"?

Vormetric's Transparent Encryption "encrypts data within your AWS instances, provides policy-based data access controls, integrated key management, and provides detailed Security Intelligence information about data access."

The main selling points are: HIPAA/PCI/HITECH compliance, policy-based access, logging/auditing, alerting. Access is host based rather than user based (which I see as a disadvantage). It also has an outright cost of $35,000.

There are a few things that these offerings all tend to mention that Vault doesn't address yet:

HIPAA/HITECH/PCI Compliance Support for the KMIP protocol Alerting AMI Images (which I guess you guys would probably attack from a Vagrant or Docker based perspective)

So, how does Vault stack up? Is Vault ready for production in an enterprise environment?

Thanks again!

armon commented 9 years ago

@Miserlou So based on that, I can say the following:

We'd like to get to the various compliance milestones, particularly PCI-DSS, but have not currently done it. We are starting with a full code audit by iSEC next month.
I'd never heard of KMIP, but PKCS11 is something we wanted to support. I'd be interested to see if there is any community demand for KMIP.
Alerting, we see this as a bit orthogonal to Vault itself. This is mostly based on ingesting the audit logs and applying some heuristics. This is something we will likely add to our commercial product Atlas.
AMI images, I think most people would prefer to configure those themselves given the binary or source. Although it would be fairly trivial for us to bake them.

So all of this said, our goal is for Vault to be competitive with all those tools in a production environment. We are already talking with several very large organizations preparing to run Vault in production and we do so ourselves as well. However, it is a young project (not even 0.2!) so it has a lot of maturing left to do.

Miserlou commented 9 years ago

Thanks for your reply, Armon!

Looking forward to the audit and PCI/HIPAA compliance. Honestly, this is really the big one for us as we work with medical and financial clients. Any idea of a time frame here?

KMIP isn't a big deal for me, I'd honestly much prefer a well documented HTTP API. However, KMIP may be good to support if you want to become a replacement of "legacy" systems because I think it's still used in some established enterprise environments.

Alerting through Atlas rather than Vault itself makes sense. What's the recommended way of consuming Vault logs in real time from an external application? For instance, integration with Graylog?

You might be right about AMIs, however, there is more to running a very secure application than just putting it on a box and forgetting about. I think that if you put out an AMI of a hardened Vault machine on the Amazon and did a bit of press, you guys could make a pretty chunk of change! Just a thought! Alternately, a guide to running a hardened Vault instance on an AMI with updates and locked down permissions and chrooting and SE Linux and all of of that good stuff would go a really long way.

Thanks again for your time! R

armon commented 9 years ago

@Miserlou The best way to consume the Vault logs is to implement an audit backend, this way Vault simply pushes the request/response logs to the target. There is no way to "pull" those out of Vault currently.

I do agree with you, we probably should create a guide or better even, just Terraform + Packer code, to build a hardened Vault environment.

Thanks for the feedback!

trevor-vaughan commented 9 years ago

Just a note regarding KMIP. This is one good reason to integrate it: https://wiki.openstack.org/wiki/KMIPclient.

eltimn commented 8 years ago

+1 for KMIP support. MongoDB's new encrypted storage engine will rely on it.

Engineering an Encrypted Storage Engine

See slide 21.

jefferai commented 8 years ago

I would hope that that's merely a first implementation of an abstract interface; something like MongoDB only really needs to be able to round trip data through an encryption service, and ought to be able to support multiple backends.

eltimn commented 8 years ago

After watching that entire video, I am under the impression they will be adding other ways to retrieve the keys after the initial release. And it's just for key management, the encryption itself is done internally.

Anyway, it's not likely that we will be using MongoDB's new encryption storage engine any time soon, so my +1 for KMIP can be disregarded.

LeDominik commented 8 years ago

Interesting, Cassandra's encryption in the Datastax's Enterprise offering also uses KMIP... http://docs.datastax.com/en/latest-dse/datastax_enterprise/sec/encryptKmipKeys.html

leeadams commented 7 years ago

Hi all (and particularly @armon),

I believe there may have been a material change regarding KMIP support since the last comment on this thread. In short VMware have released vSphere 6.5 containing a feature called 'VM Encryption' which is a hugely-powerful method for protection VM data at rest. However, VM Encryption requires a KMIP v1.1-compatible key management server to function and this may well drive very substantial demand for this technology given the scale of VMware's presence in the enterprise and datacentre generally.

More here: VMware Blog VirtualizationHowTo.com

We are implementing VM Encryption ourselves and have spoken to Vormertric, SafeNet, Hytrust and others and in short the key management solutions are extremely expensive (think tens of thousands of dollars as per the post from @Miserlou). We'd have been very interested in an open source solution, particularly, given the critical nature of the role, one that offered support or an 'Enterprise Edition', etc.

Although time is pressing for us an hence we'll likely have to go with one of the aforementioned solutions (at least for now) I'd be very interested in any KMIP developments relating to Vault.

Thanks,

Lee.

trevor-vaughan commented 7 years ago

@leeadams How are you handling key escrow? I keep nosing around but Vault doesn't have an indemic key escrow functionality so I'm not sure how useful it will be.

Imagine that you have a VM that you disable for X years but, due to various regulatory requirements, you need to be able to turn it back on at any time and recover your state.

To do this, you'll need a permanent escrow of the key that is bound to that image. I suppose that you could offload it to some other database and have a process for stuffing it back into Vault when the time is right but that seems like a lot of work for something that already stores keys.

I spoke with the Vault folks about this in the past and the current answer is to just set a stupidly long expire time on the data and move it to a different namespace.

I'm definitely interested on how people are solving this issue though and it sounds like you might have some insight.

jefferai commented 7 years ago

I spoke with the Vault folks about this in the past and the current answer is to just set a stupidly long expire time on the data and move it to a different namespace.

This doesn't sound right. Opaque K/V values never "expire" as they don't have leases.

Additionally, if you wanted to store the values separately you could use transit to store the encryption keys with the machine. So long as you don't get rid of the transit key in Vault you can recover the encryption key.

darkmars commented 7 years ago

Hi @armon, We are evaluating the Vault for usage in a PCI-DSS environment. Could you please update on the status for compliance?

Thank you in advance.

BR, Darko

armon commented 7 years ago

@darkmars We have several customers using Vault in PCI compliant environments now. Particularly with the HSM integration in Vault Enterprise, we are now partners with SafeNet / Gemalto. Hope that helps!

darkmars commented 7 years ago

@armon Thank you for the prompt reply. We are looking into using it within AWS environment, potentially by implementing HashiCorp Vault environment on AWS.

shantanugadgil commented 7 years ago

Hi,

I have posted my queries about FIP 140-2 for Consul/Nomad already, so goes the same here:

Any chance Vault will be FIP 140-2 compliant ✔️ so that it can be used in the AWS GovCloud environment? (FedRAMP requirements 🙄 )

jefferai commented 7 years ago

@shantanugadgil we're working on getting a certification for Vault regarding usage of Vault being compliant when used in an environment where the root of trust is protected via FIPS 140-2 (in other words, when using an HSM to wrap/encrypt Vault's master key).

Closing the thread at this point as it's kind of a necrothread.

trevor-vaughan commented 7 years ago

Necromancing the thread.

The root of trust is not all that needs to be protected. All cryptographic operations, including authentication, must use a NIST validated (read 140-2) cryptographic module. I don't see how a HSM, particularly one that is network-based, will work with this.

This is not just AWS, this applies to all systems under FISMA and FIPS-200 requirements.

jefferai commented 7 years ago

@trevor-vaughan It depends on what your requirements are. Not all compliance scenarios require this. (There are also some interpretation issues.)

trevor-vaughan commented 7 years ago

@jefferai Interpretations are fine but the policies are very clear.

All FISMA and FIPS-200 (Minimum Security Requirements for Federal Information and Information Systems) systems require NIST-validated cryptography. I have yet to find another public definition for NIST-validated cryptography outside of the NIST Special Publications.

jefferai commented 7 years ago

@trevor-vaughan @amanoske may be able to provide more color here.

shantanugadgil commented 7 years ago

Hi @jefferai , Thanks for the info.

No disrespect, but I don't think an "as long as it is ..." can be used when dealing with customers using AWS GovCloud and when things having to be FedRAMP authorized/ready/compliant which use any sort of crypto.

I am not an interpretation expert, so apologies if I am way off base here.

The only hurdle in all this (I think) is the use of the crypto library. I think the normal approach for Go based programs is to eventually link with a compliant OpenSSL using something like Cgo or https://github.com/spacemonkeygo/openssl ?!? Could that be a viable solution for HashiCorp tools?

Thanks and Regards, Shantanu

jefferai commented 7 years ago

@shantanugadgil You and @trevor-vaughan seem to be assuming that the only use cases for a FIPS requirement are FISMA/FIPS-200 systems, but this is not the case. We have seen many customers that have industry, internal (especially at very large enterprises), or non-US-governmental requirements around storage or encryption of master keys/roots of trust that piggyback on FIPS as a published standard. (In many of these cases, they don't want other crypto operations to be FIPS compliant due to mandated insecure aspects of it.)

Obviously whether this works for your needs is something that you need to evaluate, but the statement we're working on getting certified will suffice for many of these needs, as it depends strongly on the specific requirements, interpretations (e.g. how specific auditors for those requirements deem them to be met, or not), and so on.

FWIW, not even the U.S. government agrees -- we've had talks with many U.S. governmental agencies and many of them are not concerned with all crypto operations happening on a certified stack.

This is not to say that there could not eventually be a Vault version that uses a FIPS crypto stack for all operations, merely that one of these sets of needs is low hanging fruit and the other is a huge, costly, and time-intensive endeavor.

trevor-vaughan commented 7 years ago

@jefferai Thanks for the explanation for the record. TLDR: No using Vault on FISMA compliant systems.

@shantanugadgil I'd be interested to hear what you choose since this isn't an option for Federal systems.

jefferai commented 7 years ago

@trevor-vaughan I struggled with whether to delete your comment outright, but instead I'll simply state "for the record", as you put it, that I do not agree with your TLDR assessment of the situation, nor your assertion that Vault isn't an option for U.S. government federal systems.

shantanugadgil commented 7 years ago

@trevor-vaughan we are using KMS. Ref: https://aws.amazon.com/compliance/services-in-scope/

networkjackbb commented 7 years ago

With MySQL's Transparent Data Encryption supporting KMIP via a Key Management Server, having KMIP support in Vault would be advantageous.

jefferai commented 7 years ago

KMIP is a very, very, very large spec.

It'd be more ideal for MySQL to support fetching a key in a pluggable way.

zrml commented 6 years ago

Hi all and @armon: we just implemented KMIP client and ideally I'd like to use Vault as we provision our cluster. Why not start like every software project: one step or API at a time? It appears that there are MANY -especially in the enterprise space, interested in this ;) Thank you for giving it a serious consideration.

davwilliams commented 6 years ago

@jefferai, I know you want to retire this necrothread, but wanted to add one more comment. Agreeing with @zmrl, I think a good goal would be to implement some basic KMIP support, and there is undoubtedly demand for it. Would also help drive interest in Vault Enterprise since it covers so many other use cases besides Key Management. Or maybe add KMIP as an Enterprise-only feature.

Since PKCS#11 and KMIP are now both governed as OASIS standards and there's recognized overlap, most of the work has already been done (supporting the various management objects). It's the client-server API that's really in question.

I would recommend considering KMIP 1.1 as a baseline. In summary, there are:

39 mutable attributes that a client can assign as metadata for each key
28 client-to-server operations (i.e. create, register, rekey, revoke, locate, etc.)
2 server-to-client operations (this may net new functionality as I don't think Vault today initiates any server-to-client communication)

The message format is pretty straightforward and well-documented.

jefferai commented 6 years ago

@davwilliams Believe it or not we actually have interest in supporting enough KMIP to handle specific use-cases; it's the details that are the problem. From my understanding the binary protocol is required to be supported, JSON is optional, which means we can't rely on any given client supporting JSON, and even then the JSON format is very different from ours.

We have some ideas about how we can do this, potentially even a way forward, but it requires a lot of groundwork. So the timeline isn't soon, but I do want us to get there eventually.

davwilliams commented 6 years ago

Understood. This may be a great opportunity to design a framework built around Vault. For example, an adapter or plug-in model where someone writes a KMIP interface as an intermediary that talks standard Vault API on the backend.

Thanks for the quick follow-up!

jefferai commented 6 years ago

@davwilliams No comment :-)

melo commented 6 years ago

Hi,

arrived at this thread looking for "Vault and KMIP" support. I've read all the references to KMIP in it.

If Hashicorp plans on supporting KMIP in the future, may I suggest that you create a new issue for "KMIP support on Vault" that interested parties can subscribe and that Hashicorp can use to keep us updated?

Thank you,

innovationhub-asia commented 6 years ago

+1 for KMIP support in Vault. It would be a huge bonus when selecting the right solution. Totally willing to pay for the Enterprise version if KMIP is included.

jfjcn commented 6 years ago

+1 on this feature

sidexchange commented 6 years ago

+1 for KMIP. We're bidding on a large Enterprise modernization project that would greatly benefit from Vault having KMIP support since we are already committed to Vault as a key management solution. New thread dedicated to further KMIP discussion and updates would be great.