Migration path from Keyczar to a Python 3 compatible library

[nickbaum]

Hi folks,

Given that it looks unlikely that Keyczar will be upgraded to Python 3 (totally understandable), and other popular python packages are starting to deprecate Python 2 support, I'm hoping to find a smooth upgrade path to an actively maintained crypto library.

1. Is there a library that the keyczar authors recommend? Cryptography? PyNacl?
2. Could you provide a code sample to decrypt Keyczar AES ciphertext using said library?

(2) would be helpful in order to avoid having to re-encrypt all our data before migrating.

I'm posting this here since I suspect others might be in the same boat.

Thank you!

-Nick

[edknapp]

Let me give a little window into what's happening inside google. One of the issues with keyczar is that it was a full fork of an internal library and the two were not even fully compatible. Keyczar's toolschains, libraries, and platforms were inconvenient for google folks to develop in so the two libraries diverged. Titus Winters gave a talk at cppcon 2017 that paints a general picture of how development at google has existed during this time (https://www.youtube.com/watch?v=tISy7EJQPzI).

Fortunately, things are starting to improve. Google's internal tooling and core libraries are starting to be opensourced. Our internal version of keyczar is being rewritten as CrunchyCrypt (https://github.com/google/crunchy), which has only c++ and java jni bindings for the moment. One goal of CrunchyCrypt is make sure the library we opensource is the same library we use internally, not a fork, so that we can avoid the same fate and incompatibilities as keyczar.

This brings us to python... our internal version of keyczar uses swig, which is being replaced by clif, which currently does not support bazel (https://github.com/google/clif/issues/20). Once that support arrives, we'll be in a position to deliver python APIs in line with CrunchyCrypt's not-a-fork philosophy. For (efficient) migration, we'd likely take advantage of the built-in key versioning support of Keyczar/CrunchyCrypt and wrap keyczar in whatever CrunchyCrypt APIs we land on.

If you're solely interested a pure-python library, I don't have concrete plans at the moment. Concerns around timing attacks leads me to prefer wrapping c/c++ for crypto in python.

[nickbaum]

Thanks for the update, Ed, I really appreciate it. I'm an ex-Googler, btw, 2005-2011.

I'm glad to hear that there's a migration plan going forward, and . as long as I can get it running in a Heroku buildpack, I personally don't need a pure-python library.

I suppose the remaining concern is timing. Do you have a rough sense for when this might happen? First half of 2018 or later?

[edknapp]

I doubt anything we'll have anything soon. clif-based crypto and key-management is on the horizon, but we likely won't get to it until late 2018. It's not unlikely that a crypto-only API would come sooner, much like what we already have via jni java.

[nickbaum]

Thanks for the additional info, I appreciate it. I'll keep an eye out for any news!

In the meantime, some rough code to decrypt Keyczar ciphertext using Cryptography, PyNacl or another well-regarded Python library might be helpful for others who need/want to transition sooner.

If I get around to doing this, I'll post it here... unless someone's already done it and wants to share!

[tie84]

This is embarrassing... ... It is 2018, many dependencies for this project, people trusted you Google as professionals and you cannot just simply write this code to be compatible with both versions of Python. There is a module called six which can help with that. I saw this code and i am shocked how amateur it was written. I expected something professional but it looks like first program of student in high school. You should stop finding excuses and fix this library to work as good as it should. You should write open source code as good as commercial software to show that you are as good as you advertise yourself. Now this library is horrible and it looks that you are much worse programmers as it is widely believed.

[nickbaum]

I know this is a frustrating situation, but I hope we can all agree that attacking the maintainers of the project isn't constructive.

Given that other libraries are starting to drop support for Python 2, it's increasingly a problem that keyczar doesn't support Python 3, and it's disappointing that Google hasn't provided more organizational support for the project.

However, unless someone with the necessary expertise wants to take on the Python 3 port in their spare time, we should be grateful for folks like Ed, who is clearly putting time and effort into providing a path forward.

🙏🏻Thank you @edknapp!

[nickbaum]

@edknapp , any updates on this?

I'm starting to see more and more Python libraries update to Python 3 only ☹️

[edknapp]

Unfortunately, I've moved on to other things, the folks behind github.com/google/tink now own all these projects, you might reach out to them.

[thaidn]

Hi everybody,

I'm Thai from Tink. We're aware of Keyczar and we want to provide a migration path. The plan is not different from what Ed wrote in https://github.com/google/keyczar/issues/213#issuecomment-356145836, replacing Crunchy with Tink.

We're still blocked by https://github.com/google/clif/issues/20. I'll ping the owner and see if they can prioritize it.

[edknapp]

Note that we have a skylark extension: https://github.com/google/nucleus/blob/master/third_party/clif.bzl

If you're willing to maintain that, then google/clif#20 needn't be a blocker.

[nickbaum]

Hey folks, any news on providing a Python 3 migration path for Keyczar users?

[nickbaum]

In case it helps anyone else, we will likely use the following solution from @stackstorm to migrate from Keyczar to cryptography:

https://github.com/StackStorm/st2/blob/ddd3aba57d8cccaaaef64babd07dd48e0b68c42a/st2common/st2common/util/crypto.py