search

interagent / pliny

An opinionated toolkit for writing excellent APIs in Ruby.
MIT License
802 stars 73 forks source link

Common need wants a pattern: encrypting database fields #98

Open fdr opened 9 years ago

fdr commented 9 years ago

and with key rotation, which seems to be the place where most existing approaches fall down. @deafbybeheading was compelled to write new software because of this noteworthy defect in most or all others: https://github.com/deafbybeheading/attr_vault. Not sure how much of it is general and how much was get-the-job-done. Also, bugs.

msakrejda commented 9 years ago

+1. I think attr_vault is not mature enough yet, but we're battle-testing it, and I think with some work (and docs), it could be a great fit (though of course I'm a little biased).

fdr commented 9 years ago

Shrug. I looked around for a library with a crisp rotation angle. Null.

brandur commented 9 years ago

@deafbybeheading Hah, attr_vault sounds like a much better solution than what we could've materialized out of thin air.

Great idea guys! It would be amazing to start demonstrating some internal (/external) consistency in this area.

pedro commented 9 years ago

Hey folks,

I've been writing some Pliny docs (about time!). Check it here: https://github.com/interagent/pliny/wiki

I think encryption would make it for a great topic for the "Extending" section, if anyone wants to take a stab.

It's probably also worth mentioning Fernet – which makes me wonder, why doesn't attr_vault just let Fernet handle the actual encryption?

msakrejda commented 9 years ago

@pedro awesome! And attr_vault basically does use the same crypto as Fernet: it just omits the ttl (which is generally pretty useless when actually storing data), and the Base64 encoding (which is unnecessary since Postgres has perfectly good support for binary data--though watch out for this outstanding pg issue which has been fixed and will be in the upcoming 0.18.3). I didn't maintain Fernet compatibility since I felt it was not that important. I support "migrating" from a plaintext column already, and plan to support migrating from a Fernet-encoded column in the future. Also, given the design of attr_vault, one could actually add Fernet compatibility pretty seamlessly (essentially, you could add encryption mechanism metadata to the keys in the attr_vault keyring, and then have the library choose an encryption / decryption mechanism based on that).

But in any case, I think real progress on this issue is blocked on me actually writing some docs for attr_vault.

ys commented 9 years ago

I am using attr_vault in Limerick. I would change one thing only, the config vars using json is something I would update. Otherwise it is a great one for this.

msakrejda commented 9 years ago

@ys update how? The metadata in the keyring is important. I guess I can wrap the whole in Base64 maybe? But please file an issue on that project and propose alternatives there. In fact, here: I made one for you https://github.com/uhoh-itsmaciek/attr_vault/issues/14

ys commented 9 years ago

Let me take a look and come with an idea:) This is my only issue with it.

pedro commented 9 years ago

@uhoh-itsmaciek got it! attr_vault looks super promising then. I take you folks are already using it in prod?

If you want to write some docs in the repo I can take a stab at the Pliny guide, lemme know!

msakrejda commented 9 years ago

@pedro yeah, we're using it in Shogun (and possibly other places) and other than the ruby-pg issue mentioned above and my lack of docs, it's been pretty seamless. I'm doing some cleanup and docs now.

msakrejda commented 9 years ago

@pedro btw, any tips for documenting a Ruby library like this? My first pass will just be going over usage in the README, but wondering if you know of libraries with clear, simple, idiomatic Ruby docs. I guess in my experience, I usually go for the README of whatever gem I'm using--maybe that's enough here?

pedro commented 9 years ago

@uhoh-itsmaciek can't claim to know much about this, but excon and dotenv come to mind when I think of good docs I used recently-ish!

Most of the gems we use are unsurprisingly well documented – probably some correlation there. But one example that comes to mind on bad docs is rr: they changed it to become a mix of readme and files inside the repo and that confuses the hell out of me.

So yeah, big +1 to just keep it all under README.md. Just went for a wiki in Pliny because I was thinking of it in terms of articles that would explain the different patterns and point people to more docs on them. Still gotta consider how is this going to fit with the repo readme, as it's certainly no replacement for it.

bjeanes commented 9 years ago

But one example that comes to mind on bad docs is rr

Yeah and the first result in google is the wrong repo, which always gets me too. So bad >.<