Replace BigInt based elliptic curve library

randombit commented 7 months ago

Botan 3.5.0

In this release pcurves is really just used for hash to curve

[x] Initial pcurves (point arithmetic, fixed curve params) - that's #3979
[x] Deprecate all the functionality that existed just to support elliptic curves using BigInt, eg mod_sub, ct_reduce_below, many more.
[x] ~Support for providing parameterized curves, where we eg compute Montgomery params at runtime. This is required not just for user provided/application specific curves but also I don't think it's worthwhile to provide the fully parameterized/hardcoded support for obscure curves like secp160r1.~ In the short term, application curves, secp160r1, etc fall back to the currently used BigInt code

Botan 3.6.0

In this release, we tie together EC_Scalar/EC_AffinePoint to pcurves so that everything goes fast :rocket:

[x] Convert EC keys internally to store EC_Scalar and EC_AffinePoint instead of BigInt/EC_Point (#4203)
[x] Bridge between EC_Scalar/EC_AffinePoint and pcurves (#4143)
[x] Add EC_Scalar and EC_AffinePoint types and implement algorithms using them #4042
[x] RFC 6979 will need some work and could become more optimized by having dedicated EC_Scalar version to avoid EC_Scalar<->BigInt conversions. (Now in #4042)

Bumped to 3.7.0

Work originally planned for 3.6.0 but bumped to 3.7.0

[ ] Add new constructors to EC keys, deprecate old constructors
[ ] Technical debt patrol - remove duplicated logic between the various types
[ ] Start removing stuff replaced by pcurves, eg CurveGFP_NIST (we can just use Montgomery for everything)

Botan 3.6.0 or later. Nice optimizations / cleanups but not critical

[ ] Faster mul2_vartime. ~Based on some reading and experimentation best option for parameters of our size is a wNAF with w==4 which should hit on average 80% dual 0s and requiring a table of 80 group elements.~ JSF or bust it seems
[ ] Better variable base mul algorithms. OpenSSL ECDH uses https://link.springer.com/content/pdf/10.1007/3-540-45664-3_20.pdf
[ ] co-Z mul? https://eprint.iacr.org/2011/338.pdf
[ ] Faster doubling. We saw a huge bump for ECDH and ECDSA verify from #4221 / #4225 / #4257, this maybe the lowest hanging fruit right now. Maybe dbl-2004-hmv
[ ] Avoid possibility of a doubling in the fixed window mul due to blinding (see https://github.com/randombit/botan/pull/3979#discussion_r1624604450)
[x] Optimize RFC6979 further - it consumes 50K cycles which really hurts especially for P-256
[x] More curves possibly? NUMS (#4251), P-192 (#4190), P-224 (#4218)
[x] Optimize bytes_to_words kind of comically inefficient right now
[x] Figure out how to speed up inversions. Either searching for addition chains at compile time and/or providing a way of conveying a specific addition chain where a good one is known. Inversions are 30-40% of the entire cost of ECDSA signature generation right now, and 15-20% of the cost of ECDSA signature verification. #4209 #1479 #4211 #4215 #4213
[x] Specific field reduction support for P-256, P-384 (#4147), secp256k1 (#4113), NUMS (handled similarly to secp256k1), sm2p256v1 (Solinas prime so handled like P-256 #4240)

reneme commented 5 months ago

We should consider compile time configuration of which curves get instantiated (as pointed out here). Much like the compile-time configuration of having "kyber" and/or "kyber_90s", for instance.

Adding one build module per curve should do the trick, and probably provides the best affordance for users. On the other hand, it produces quite a bit of boilerplate due to subdirectories, info.txt files and code machinery.

Alternatively, I could see this as a dedicated compile-time switch: Like:

./configure.py --enable-elliptic-curves=all (the default)
./configure.py --enable-elliptic-curves=p256,brainpool256 (a restricted set)

and then simply #ifdef the instantiations in pcurves.cpp accordingly.

randombit commented 5 months ago

The --enable-elliptic-curves approach is potentially confusing since even if you disable the pcurves module entirely, the elliptic curves still exist. All that ends up happening is you fall back to the slower BigInt code. [*]

[*] I mean right now ECDSA etc still use BigInt, even after both #3979 and #4042 land. There are several more steps here until the whole thing hangs together. But in the end state, we'll have fast ECC for specific curves, plus fallback code for weird curves, application curves, etc. If you disable the fast curve, it doesn't disable the curve, just disables it going fast.

That said there may be environments where the code size becomes an issue. I tried out a module per curve and it was not so bad. There are likely to not be more than ~5 new curves added here over time so I don't think it's an issue in an ongoing way.

In the end we could consider also having a way of disabling the slowpath BigInt stuff, which would benefit environments that are using just P-256 or something.

reneme commented 5 months ago

Nice! I saw that you split the curve into compile-time modules. Indeed the boilerplate isn't so bad. Also provides a nice place to keep curve-specific special cases. :+1:

I added a few minor suggestions, in which the type -> "Internal" might need some extra attention. Perhaps have a new module type "Feature" (which might also be better suited for "Kyber" vs. "Kyber90s", and similar situations...)?

Honzaik commented 2 weeks ago

Hello,

I am new to this library so I might be missing something, but from what I understand, you are deprecating EC_Point and replacing it with EC_AffinePoint. My concern is, that currently I dont see currently how to add two distinct EC_AffinePoints. Of course, for Diffie-Hellman all one needs is interface that takes a scalar n and calculates n*P for some point P. However, in other applications you may want to add distinct points.

So my question is, is there a way to add two EC_AffinePoints?

Thank you in advance.

randombit commented 2 weeks ago

@Honzaik this is not currently supported simply because the interface of EC_AffinePoint was quite intentionally done as the absolute minimum required to implement the relevant algorithms. This is to a large extent a resposne to EC_Point which has a huge number of very specialized interfaces, many of which make it difficult to implement EC_Point using any other approach than exactly how it is currently implemented.

In the short term the answer is if you want point addition use EC_Point - it’s deprecated but thats just an advisory that it will be removed down the line (next major release in ~~~ 2027)

It’s very likely EC_AffinePoint will gain point addition. I debated adding it during the initial implemenetation and then decided to wait until it’s actually in use. That will come anway when SPAKE2 is implemented, since that requires point additions.

I would be curious as to what kind of protocol you are implementing.

Honzaik commented 1 week ago

@randombit thank you for the explanation. My use case is similar to SPAKE2, i.e., some PAKE-adjacent constuctions. Specifically, PAPKE (https://eprint.iacr.org/2019/199.pdf) which can be used to build PAKE requires addition of different points. Similarly, the M2F construction from https://eprint.iacr.org/2023/295.pdf also requires addition.

These constructions also need hashing to curve, but that is already implemented (for prime curves), however, I'd also note that the "uniform hashing" construction (https://www.ietf.org/archive/id/draft-irtf-cfrg-hash-to-curve-16.html#section-3-4.2.2) also requires adding two points.

Don't get me wrong, I am doing just a proof of concept implementation and all these use cases basically boil down to PAKEs so I understand I am a tiny minority of users that need it. At the end, I understand why a lot of crypto libraries try to minimize the opportunity for their users to shoot themselves in the foot and power users can in the end fork it and expose the hidden functions (with a little bit of work). On the other hand, I feel like leaving the addition implemented doesn't pose that much of a risk (maybe as some sort of a “hazmat” layer, like in python cryptography).

randombit commented 1 week ago

It's not specifically about preventing users from shooting themselves in the foot. TBH working at this level -- using raw elliptic curve points to define a new protocol -- you had better know what you are doing anyway! It was more that the interface of EC_AffinePoint was defined to be just what was required to implement the various elliptic curve algorithms already included, and it so happened that we did not need at this time to expose point addition to the rest of the library, thus it was not added.

Point addition will be added when SPAKE2 is added, or perhaps I'll just add it for 3.7.0 right away since addition plus point negation probably unblocks most of the basic protocols one might want to implement.

randombit / botan