Spongy Castle: is it obsolete?

rtyley commented 7 years ago

Spongy Castle was created back in 2011 because the Android platform bundled an old, restricted subset of Bouncy Castle. Simply adding an updated version of Bouncy Castle to your app resulted in class-clashes and exceptions - so you needed a repackaged & renamed version, like Spongy Castle.

Why might Spongy Castle be obsolete?

As mentioned in the book Android Security Internals by @nelenkov, the org.bouncycastle packages in Android were renamed to com.android.org.bouncycastle starting with Android 3.0 (see also https://github.com/open-keychain/open-keychain/issues/1676 and Android Platform commit 0ac85ead96f - note that the version of BC bundled with Android 4.0 was still a restricted subset). So using Spongy Castle may no longer be necessary, if you can just include an up-to-date version of Bouncy Castle directly.

There are also alternative crypto libraries to consider, like Facebook's Conceal, Google's Conscrypt, libsodium-jni, etc.

Why might Spongy Castle not be obsolete?

pre-Android 3.0 devices are still in active use. There are higher areas of use in poorer countries, and those people still need secure cryptography. Signal (not a SC user, so far as I'm aware) currently still supports Android 2.3 and up.
even on post-Android 3.0 devices, device manufacturers are not above carelessly bundling libraries, it's possible that Bouncy Castle may still be bundled on some obscure devices.
Although the version of Bouncy Castle bundled with Android has a changed package name, it still has "BC" as the provider name, leaving some ambiguity as to the choice of implementation when adding your own copy of Bouncy Castle to the app and choosing "BC" as your provider (thx to David Hook for passing on this point)

So Spongy Castle may still be necessary, if you really do need the functionality of Bouncy Castle on older devices. However it takes non-negligible effort to maintain Spongy Castle, so I'm using this issue to solicit feedback from people on whether they do, or do not, need further releases of Spongy Castle, and also to answer the users who ask "Why hasn't the latest release of Spongy Castle come out yet?"

Why do releases of Spongy Castle lag so far behind Bouncy Castle?

Version	Bouncy release	Spongy release	Lag in days
1.50	3 Dec 2013	26 Jan 2014	54
1.51	25 Jul 2014	26 Jul 2014	1
1.52	2 Mar 2015	15 Jun 2015	105
1.53	12 Oct 2015	18 Oct 2015	6
1.54	29 Dec 2015	29 Dec 2015	0 (thanks to advance warning from @cwgit)
1.55	18 Aug 2016	-	...
1.56	23 Dec 2016	-	...

The Bouncy Castle project has a suite of nearly a thousand tests, and with every new release of Spongy Castle I want to make sure that those tests all pass - I have to make sure I haven't broken anything. In particular, I want to make sure that those tests pass in public CI (eg Travis CI) so that people can reproduce my work.

Unfortunately, this is surprisingly difficult:

The Bouncy Castle project itself has no public CI setup, and they've not been able to share their private CI infrastructure with me.
Many tests in the suite trigger javax.crypto.JceSecurity.verifyProviderJar() checks that will only pass with one of these two problematic options:
- Obtain a JCE Code Signing Certificate from Oracle, to sign the Spongy Castle provider for the purpose of the tests - note that the signature is not required for Spongy Castle to operate on Android, just for the tests to pass. I've not yet attempted to go down this path.
- Avoid the signing requirements during tests by using OpenJDK, specifically OpenJDK 7 or earlier - which is no longer available on the current version of Ubuntu.
Generally, with every new release, a new set of tests fail for other reasons too, and investigating those takes time. Some of the fixes I've found I've been able to contribute back to the upstream Bouncy Castle project.

Sunset on a Spongy Castle?

Given the inevitable obsolescence of Spongy Castle, at some point there will be literally no reason to keep on devoting the significant chunk of time it takes to cut a release! That time may not have arrived yet - but for my own sanity, I'd like to know when it does, so I can use my time to do other stuff.

So, please vote on this issue:

:+1: to say Spongy Castle is obsolete
:-1: to say you still need it

...and if you could add a comment, explaining why you need it, and linking to your project, that would be great and hold much more weight with me.

dschuermann commented 7 years ago

👍

martijno commented 7 years ago

Going to test our setup by bundling with regular Bouncy on Android. BTW thanks for you hard work @rtyley!

akumaigorodski commented 7 years ago

These two actively developed projects depend on spongycastle (and my android wallet in turn depends on them):

https://github.com/bitcoinj/bitcoinj https://github.com/ACINQ/bitcoin-lib (this one already needs v1.55 so spongycastle branch lags behind master for now)

rahulmcs commented 7 years ago

I am working on a major UK banking app that has more than a couple of million active users and we would be definitely interested in having an upgrade to the latest version of BouncyCastle through Spongy. In the longer term, we will be looking at alternate options but near term an upgrade would be very helpful.

rtyley commented 7 years ago

am working on a major UK banking app that has more than a couple of million active users

Thanks for your response @rahulmcs - given that you're working on a commercial project, would the bank you're working for be interested in sponsoring me for a day of work so I can get this release of Spongy Castle out?

akumaigorodski commented 7 years ago

I'm willing to send 0.1 BTC to at least partially compensate for your time. Please provide an address if that's fine with you. anton.kumaigorodskiy@outlook.com

rtyley commented 7 years ago

Many thanks for your kind offer @btcontract. My bitcoin address is 1HTBHTjsZddo8PJU5eT6EcNk4mzLU6m5Zf - all funding is much appreciated, it would probably take about 0.3 BTC for a full day.

akumaigorodski commented 7 years ago

Done. Hope others interested will join and we'll get a release sooner than later :)

Skywalker-11 commented 7 years ago

👎 if you take a look on how providers and manufacturers are handling android updates you have to buy a new phone at least every two years to use an android that still has an accepable security level. If you don't have to depend on the normal update cycle but instead depend on your own crypto lib you can assure that at least the communication between server and client is secure. I'm too working on porting bouncycastle to the current version with aar libs. The buiild is working (including the mail part) only problem are the tests 😄 .

tsombrero commented 7 years ago

+1 thank you so much for the Spongy effort. I've managed to get BC 1.56 working after realizing it was necessary to call Security.removeProvider(BouncyCastleProvider.PROVIDER_NAME); to remove the built-in BC before calling Security.addProvider(new BouncyCastleProvider());.

rtyley commented 7 years ago

Release v1.56.0.0 is now available on Maven Central - more details here at : https://github.com/rtyley/spongycastle/releases/tag/sc-v1.56.0.0

thejoelpatrol commented 6 years ago

Like @tsombrero said, the "BC" issue is an annoyance, but not a show-stopper for using regular BouncyCastle, at least in my case. I thought I would have to use SpongyCastle for my new project, but I've managed to use regular BC by removing Android's bundled provider like he said.

jbuhacoff commented 5 years ago

Hi everyone, I made a tool called mybc for building your own customized BC library, so you don't need to wait for the next SpongyCastle release. I'm using it in my Android app. Currently it's building BC version 1.62 for JDK 1.5 successfully. Links: mybc on GitHub and mybc on NPM. If it helps please star the repo!

swapnilgt commented 4 years ago

We are shipping some SDKs for our clients to consume them inside their apps. When building with spongycastle, the SDK size almost inflates by ~3 MBs. For spongycastle, the proguard rules for thinning out the unused parts used are very generic:

-keep class org.spongycastle.** {*;}
-keepclassmembers class org.spongycastle.**

If I remove these rules, proguard thins out everything, causing a runtime exception where the SC provider could not be found. Can someone guide with more specific proguard rules so that my SDK size does not inflate a lot?

PS: Removing the SC as a provider and using the method KeyFactory.getInstance(KEY_GENERATOR_ALGO) runs fine with several devices. Not sure, if this is a safe way to proceed based on the comments above.

bcgit commented 4 years ago

I'm not fully qualified to answer this one, but I think I can guess what the problem would be - the provider architecture uses reflection to build the services, progaurd is not going to be able to deal with that sensibly. If you had a look at the source of the provider and the lookups you could work out a list of required classes which would help you minimise the size of the jar.