OQS goal: non-committal research or production use?

baentsch commented 9 months ago

Following up on a discussion about the goal of OQS this issue is to raise the question whether OQS shall remain limited to research-only use as per the currently published goal

aims to support the development and prototyping of quantum-resistant cryptography

and limitations

OQS is intended for prototyping and evaluating quantum-resistant cryptography

WE DO NOT CURRENTLY RECOMMEND RELYING ON LIBOQS OR OUR APPLICATION INTEGRATIONS IN A PRODUCTION ENVIRONMENT OR TO PROTECT ANY SENSITIVE DATA. This project is meant to help with research and prototyping.

or whether it should strive to become code that users may be able to rely on at some time -- and if the latter, when.

@dstebila suggested I should propose text how to phrase this, but as a person not employed by a company that might benefit from such change of mission and considering that apparently all TSC members (again, except me) are within research organizations, I do not feel in a position to do so. I'd rather solicit input whether the TSC is interested at all in the goal of making OQS more geared to production use -- particularly as the companies funding PQCA are starting a parallel project launched geared to the development of high-assurance, production-oriented code without the "technical debt" of OQS.

By retaining the goal of "research-/evaluation-only" for OQS

the integration of code with unclear licensing or APIs that have not been tested for actual usability is much easier.
OQS can retain its current "casual" update pace
doesn't have to take at heart security considerations (e.g., FIPS) or stringently have to follow upstream security updates

Unsure whether tagging @open-quantum-safe/tsc @ashman-p @thb-sb (the latter two apparently not members of this org?!) is necessary to get your attention.

dstebila commented 9 months ago

Unsure whether tagging @open-quantum-safe/tsc @ashman-p @thb-sb (the latter two apparently not members of this org?!) is necessary to get your attention.

Norm and Thomas's invitations to @open-quantum-safe/tsc are still pending; once they accept that should suffice.

Going into meetings now but I'll respond later today trying to explain how I see the relation between OQS and PQ Code Package.

dstebila commented 9 months ago

How I see PQ Code Package

The goal of PQ Code Package is to produce high-assurance source code implementations of individual standards-track PQ algorithms. PQCP will start with Kyber / ML-KEM, and if that goes well, would consider expanding to Dilithium / ML-DSA, Falcon, and SPHINCS+ / SLH-DSA, probably not sooner than a year from now.

The intended adopters of PQ Code Package are cryptography library authors who want source code for these algorithms that they can incorporate into their own library, without introducing a new binary dependency. For example, Mozilla wants to pull in a high-assurance implementation of Kyber, but needs to add that to their source repository in a way that they have control without adding a binary library dependency. I had originally anticipated that OpenSSL would also want to take this approach, although the recent discussions around CLA requirements throw a wrench in that.

Very early in the brainstorming of PQCP there was raised the possibility of making a "Kyber OpenSSL 3 provider" based on the Kyber implementations in PQCP, but that hasn't been discussed much lately. There haven't really been other discussions around distributing binaries from the PQCP initiative.

How I see OQS

OQS started off as "software for prototyping quantum-resistant cryptography", and in our OQS visioning exercises last year the group made it clear that they wanted a dual mandate going forward: a production-ready library for standards-track algorithms, and a platform for continuing to support prototypes and experiments for emerging PQ algorithms.

For OQS to achieve the production-ready side of the intended dual mandate, it will need high-assurance implementations: these would come from the PQ Code Package, once they're ready. OQS will probably be the first consumer of PQCP implementations, and indeed Pravek is already laying some groundwork for this with trying to understand how to bring the Kyber implementation from libjade into liboqs.

Directly comparing PQCP and OQS

OQS (and OQS Provider) would be a multi-algorithm full-featured suite of post-quantum cryptography, distributed in binary form. The production-ready track of OQS would be just that: production-ready, and would use the highest quality algorithm implementations we can get, coming from the PQ Code Package where available. People using the production-track version of OQS would also be able to use the experiment-track version of OQS in their testing infrastructure to see how emerging non-standardized PQ algorithms perform.

PQCP has standalone implementations of standards-track PQ algorithms, written to be production-ready from the start. Focus solely on Kyber / ML-KEM for probably at least a year. Current intention is to distribute only as source code, not as binaries, but not inconceivable that there could be single-algorithm binaries available (e.g., an ML-KEM OpenSSL 3 provider).

To some extent I view the relationship between OQS and PQCP as similar to the current relationship between OQS and PQClean. But compared to PQClean, PQCP will be narrower in terms of algorithms (initially just Kyber / ML-KEM), more focused on high-assurance and formal verification, and have a bigger community around it.

I don't think a PQCP-based ML-KEM OpenSSL 3 provider would obviate demand for OQS Provider, as OQS Provider would support more algorithms and probably more functionality than a single algorithm provider.

Production-track OQS

The OQS visioning exercise laid out a clear desire for a dual-mandate future for OQS: a production-ready library complemented by contiuing to support experimentation with new algorithms.

I have at least three key questions on getting to production-ready:

What criteria would a production-track version of OQS need to meet?
How do we organize the OQS codebase to achieve the dual-mandate of a production-track version of OQS and an experimental-track version of OQS?
How do we get from where we are now to there?

I don't have complete answers to any of these questions, these are things we have to work on as a group over the coming months. For question 1, I at least know a few factors we will have to consider:

Release process
Security responses
Code review - internal and external

I think the forthcoming external review of OQS Provider will be really useful information feeding into this process, giving us an initial indicator of how close/far we are.

baentsch commented 9 months ago

Thanks, Douglas. Makes sense. Your questions 1-3 are a good start to the discussion. I'd add

Which algorithms should receive which "production-readiness rigour"?

Would it be sensible to discuss them within this "TSC" subproject, or publish them more widely in the OQS discussion board? Maybe with some proposals, already? My "getting started proposal" would be initial agreement on which config options have to be set to trigger "production-readiness tests" and what should those be (KAT/interop testing, memleak testing, CTT, license checks, code audits, etc)? Right now, I'd argue to keep the target as small as possible by setting this to "CMAKE_BUILD_TYPE=Release OQS_ALGS_ENABLED=STD OQS_DIST_BUILD=OFF OQS_USE_OPENSSL=ON OQS_OPT_TARGET=generic", possibly limited to arch=x86_64.

I think the forthcoming external review of OQS Provider will be really useful information feeding into this process, giving us an initial indicator of how close/far we are.

I wouldn't hold my breath for this: The task was to unearth coding problems in oqsprovider, not in liboqs.

dstebila commented 9 months ago

Thanks, Douglas. Makes sense. Your questions 1-3 are a good start to the discussion. I'd add
4. Which algorithms should receive which "production-readiness rigour"?
Would it be sensible to discuss them within this "TSC" subproject, or publish them more widely in the OQS discussion board? Maybe with some proposals, already?

In the end it would be the TSC to make a decision, but I think it makes sense to start off the discussion in an org-level discussion board. Go for it!

baentsch commented 9 months ago

Go for it!

Done: https://github.com/orgs/open-quantum-safe/discussions/1689

Will take it as a litmus test as to whether PQCA is pure marketing or real (contribution and usage) interest.

baentsch commented 7 months ago

A separate discussion triggered me to review what's discussed here and it's awfully silent: All input so far comes from non-corporate entities. Does this mean all corporate entities don't care and/or don't intend to use OQS for anything else than research? If that were the case, we could cut the discussion short. It would also mean we simply merge things like Stateful hash-based sigs without further consideration to "productive use" concerns.

Prior to the next meeting where this topic is scheduled for discussion may I therefore suggest to have such input added at least by TSC members, Microsoft (@christianpaquin ) AWS (@brian-jarvis-aws ) IBM (@bhess ) Cisco (@ashman-p ) SandboxAQ (@thb-sb ):

What's their company's intended or actual use of OQS?
What (non-functional) requirements does this use prescribe?
Any further things such use demands?

Also tagging @beldmit for a RedHat/Fedora perspective (I know, you're not a member of PQCA, but neither am I :-) I'd personally value your perspective. Please forward a link to/tag in this issue anyone else you may know having an opinion about/interest in this.

@dstebila @ryjones Please also consider tagging other people that participated in the last TSC meeting (again a quick reminder for meeting minutes...).

christianpaquin commented 7 months ago

I agree with Douglas' vision of both PQCA projects described above. I can't speak on behalf of my full organization, @baentsch, and I don't yet have all the answers you are seeking, but that shouldn't prevent us from having a dual production-ready / experimental mindset (like we use to have in OQS with two similar build targets). For me, production ready means using the "best" implementation we have access to, for a (potentially draft) standardized algorithm (NIST or in some other venue, e.g. Frodo in ISO), using standardized KATs, and tests. People could further reduce our list with build filters. "Experimental" would be everything else; I would still fix the bar to only incorporating "serious" algorithms (i.e., considered by a standardization effort, backed by a recognized research team, etc.)

Raytonne commented 7 months ago

Can't speak for others, but we use it for research/evaluation-only. This allows us to experience a wide variety of encryption algorithms ASAP.

brian-jarvis-aws commented 7 months ago

may I therefore suggest to have such input added at least by ... AWS (@brian-jarvis-aws ) ...

We use OQS for interoperability testing against our own implementations.

For example, in s2n-tls we use OQS-OpenSSL as an interop tests target for our own PQ-Hybrid TLS implementation. We plan to move to the OQS OpenSSL3 provider since OQS-OpenSSL is discontinued.

It's a similar story for some work we are doing to add AWS-LC support to strongSwan. We use the strongSwan liboqs plugin to perform compatibility checks against the plugin we're developing.

Another similar story for SSH. We added PQ-Hybrid key exchange support to the SFTP implementation used in AWS Transfer Family. This is built using the PQC in AWS-LC, but we're performing interop tests using OQS-OpenSSH.

bhess commented 7 months ago

may I therefore suggest to have such input added at least by TSC members, ... IBM (@bhess ) ...

I can't speak for the full organization.

The main use case has been to evaluate NIST PQC pre-standard algorithms. This will continue with the new PQC/DSS onramp phase where members of our team at IBM Research contribute as part of three submissions. There is willingness to help adding such on-ramp submissions as part of "experimental" oqs to allow their evaluation.

A second use case is interoperability testing especially with the OQS test server.

A third use case is more generally related to software supply chains with dependencies on open source components. Looking at the "external users of OQS", there are dependencies on OQS that to me at least blur the line between experimental and productive use. I am convinced that PQC/QSC is too important to leave the risk of using "experimental-only" components. Therefore I support @dstebila's vision of a dual-mandate for OQS where the relationship between PQCP and OQS will be important. This reliefs OQS from owning and maintaining crypto-implementations and gives its sister project PQCP the mandate to provide such higher-assurance implementations that can be consumed by OQS.

baentsch commented 7 months ago

I can't speak for the full organization.

Hence my request to disseminate this further to people in your organization that may have a "beyond research" perspective.

This reliefs OQS from owning and maintaining crypto-implementations and gives its sister project PQCP the mandate to provide such higher-assurance implementations that can be consumed by OQS.

This IMO doesn't make sense: Why add OQS as another layer (that will require certifications if part of a product handling crypto material) when one could simply add certified PQCP code to productive consumers (such as openssl) straight?

beldmit commented 7 months ago

We intend to use a subset of liboqs as a production-ready part of our distribution and plan contributing in various aspects (and already contribute)

baentsch commented 6 months ago

Another aspect here.

baentsch commented 5 months ago

As per discussion in TAC document the definition of "production use" by PQCA (@hartm) is that code is used productively, not that PQCA actually endorses or vouches for this in any way. With this understanding, this issue can be closed: OQS is use productively and the only question to be resolved is which quality criteria (functional, e.g., recommended build options and non-functional (e.g., process & procedures) OQS should adopt for which sub projects, i.e., answering issue #2.

baentsch commented 4 months ago

Pointing to a discussion regarding this topic in a PR: https://github.com/open-quantum-safe/liboqs/pull/1832#issuecomment-2222654090. @hartm, would you agree to continue the discussion regarding "production deployments" here instead of an "innocent PR"?

hartm commented 4 months ago

Uh, sure! I'm happy to talk wherever people want.

baentsch commented 4 months ago

Uh, sure! I'm happy to talk wherever people want.

Great. You write in the comment above

Who is using OQS in production deployments? And would they be upset at API changes?

While this may be an interesting question given that we explicitly tell people to not do this --and this issue is to discuss whether or not to change this-- the question in this context is wrong: It should be "Who is relying on OQS API stability?"

The answer to that question is "Everyone who integrates it.". And that's quite a few people. Any API change causes work to them, so it should at least be well documented.

As you stated somewhere to also work in research, how would you like it that your research project, e.g., some PQ performance measurement app using liboqs, suddenly fails to build due to an API change, possibly right before the paper deadline? Clearly no "production deployment" but a legitimate use -- that OQS explicitly wants to support. My goal is to minimize such "inconvenience" -- and as you rightly point out, this is a sign of a more mature project. So let me add that to the lifecycle document.

hartm commented 4 months ago

The answer to that question is "Everyone who integrates it.". And that's quite a few people. Any API change causes work to them, so it should at least be well documented.

For many reasons, it would be good to maintain a list of folks who "integrate" OQS and are willing to publicly acknowledge it. We can put something together but will need help from the community in keeping it up to date and letting us know about deployments.

As you stated somewhere to also work in research, how would you like it that your research project, e.g., some PQ performance measurement app using liboqs, suddenly fails to build due to an API change, possibly right before the paper deadline? Clearly no "production deployment" but a legitimate use -- that OQS explicitly wants to support. My goal is to minimize such "inconvenience" -- and as you rightly point out, this is a sign of a more mature project. So let me add that to the lifecycle document.

If I were doing research, I'd most likely just end up using the old version on short notice and not bother updating until after the paper deadline. I don't think this would be a big issue. I could just update later on.

The nightmare scenario is if the APIs get updated and someone has a production deployment where they use an old version (with the old APIs). Now suppose there's a big security bug fix that needs to happen ASAP, but it's only deployed for the new version of the software. Now this "someone" is in a lot of trouble because they would need to do a huge update on their software to deploy a security fix that needs to be deployed immediately. This is the kind of scenario we'd like to avoid.

With all this in mind, it's still a good idea to include LTS/API support in a lifecycle document. APIs can be changed by a mature project (and they are) but this should be telegraphed to the community well in advance.

SWilson4 commented 4 months ago

The answer to that question is "Everyone who integrates it.". And that's quite a few people. Any API change causes work to them, so it should at least be well documented.

For many reasons, it would be good to maintain a list of folks who "integrate" OQS and are willing to publicly acknowledge it. We can put something together but will need help from the community in keeping it up to date and letting us know about deployments.

We do have lists of projects and papers which make use of OQS. As far as I know, these are maintained on a best-effort basis, but they are a good starting point.

hartm commented 4 months ago

Oh, nice! Thanks @SWilson4 for sharing. That is a lot more than I expected.

How would you feel about eventually migrating this to the OQS github so that folks could issue PRs when they start using OQS?

hartm commented 4 months ago

Knoweldge of this list would change my comments about API changes to suggest more caution, but you all justified things pretty well in your explanations (and it's not my decision!).

SWilson4 commented 4 months ago

How would you feel about eventually migrating this to the OQS github so that folks could issue PRs when they start using OQS?

OK by me, I think that makes sense.

dstebila commented 3 months ago

Tying into Roadmap discussion

baentsch commented 3 months ago

I think the forthcoming external review of OQS Provider will be really useful information feeding into this process, giving us an initial indicator of how close/far we are.

Just for the record, it didn't. But time passed and now there has been a review of liboqs itself courtesy TrailOfBits: Any lessons from that with a reading on this issue, @dstebila ? Please also see the request to share the results. In addition, just for the record/reference oqsprovider now goes through a kind of internal/"friendly" LF-security assessment that already did identify and surely will keep adding many areas for concrete improvement. IMO it would be beneficial to do this also for liboqs (but maybe only after the recommendations by TrailOfBits have been incorporated).

baentsch commented 3 weeks ago

One more data point providing input to this question: https://github.com/IETF-Hackathon/pqc-certificates/pull/165#issuecomment-2453506471 clearly documents that OQS is losing its value even as a testbed for others: Is it possible that OQS is trying to be too much for everyone ("productive" code for companies, "prototyping" testbed for research & testing) and now loses its edge to be good at any one thing?

This is to suggest considering focusing again on the research & prototyping aspect, speeding up development again -- but being fully upright about this to users (and PQCA marketing to be more careful (or even no longer at all) pushing their "product readiness" message).

baentsch commented 3 days ago

Somewhat a repetition of arguments above, but triggered by the question of @dstebila allow me to link this here regardless: https://github.com/open-quantum-safe/liboqs/issues/1894#issuecomment-2497029224

--> One possible way forward to address this issue may be by wrapping all standard algorithms (like SHA3) from OpenSSL and keep focusing on "research and prototyping": Be a useful test bed, discover problems in new NIST algorithm proposals, check/improve algorithm performance, etc.

open-quantum-safe / tsc