Open mouse07410 opened 4 months ago
@baentsch I strongly suggest changing the defaults (at least for now) in oqs-templates/generate.yml, and disabling all the signature algorithms except for ML-DSA.
That'd be pretty drastic, no? Would you suggest also disabling the hybrid and composite variants, @mouse07410 ? @beldmit: What'd be your position: You seem to have taken to using some of the (now) "historic" variants of PQCrystals algs: Would you also be OK if we'd disable them by default? Any objections, @dstebila ? Researchers can always re-enable algs they need, but normal users should not be negatively impacted by shenanigans like these, so I currently tend to support this proposal.
I am going to enable "old" Kyber for Fedora 41 (rawhide) at least for now for interoperability's sake for liboqs and oqsprovider
@baentsch I strongly suggest changing the defaults (at least for now) in oqs-templates/generate.yml, and disabling all the signature algorithms except for ML-DSA.
That'd be pretty drastic, no? Would you suggest also disabling the hybrid and composite variants, @mouse07410 ? @beldmit: What'd be your position: You seem to have taken to using some of the (now) "historic" variants of PQCrystals algs: Would you also be OK if we'd disable them by default? Any objections, @dstebila ? Researchers can always re-enable algs they need, but normal users should not be negatively impacted by shenanigans like these, so I currently tend to support this proposal.
I think it would be a bit drastic to disable everything except ML-DSA. I'm not sure what the right combination is, but hopefully we can get a reasonable mix in. This is where an "adopter advisory committee" would be helpful...
That'd be pretty drastic, no?
Given that the only working alternative seems to be disabling the oqs-provider
altogether - probably not that drastic. Of course, I'm all ears to hear what options you can recommend.
Situation:
What currently works: I've successfully tested the following (using modifications proposed by @iyanmv above):
Would you suggest also disabling the hybrid and composite variants, @mouse07410 ?
The main reason for disabling some of the algorithms/variants is to make TLS work without removing oqs-provider altogether. Thus, if (as it appears to me) hybrid and composite variants aren't automatically disabled by modifications defined here - then no. If they are disabled - then yes.
I'm not sure what the right combination is, but hopefully we can get a reasonable mix in.
Offhand and IMHO, whatever from the NIST standards that fits TLS well. So far, that's Falcon and ML-DSA.
I don't know the scope of enabling/disabling algorithms. IMHO, it would be great if it were possible to separately define what algorithms are available by the provider to use, and what subset of those is advertised in TLS. In that case, I'd have everything enabled in the provider, but only the NIST variants (except SPHINCS+) advertised in TLS ClientHello.
it would be great if it were possible to separately define what algorithms are available by the provider to use, and what subset of those is advertised in TLS
That'd be a possible compromise indeed, but if memory serves, openssl
now simply advertises all sig algs it finds providers to, well, provide. Admittedly, it was me adding this logic to openssl
, so I feel at liberty to also change it again :-) @levitte: What'd be your take/recommendation in this regard? We could add a property "OSSL_CAPABILITY_TLS_SIGALG_ADVERTISE" or so to facilitate this.
This is where an "adopter advisory committee" would be helpful...
I'm not sure another committee would help here: This simply seems to be a technical limitation by existing infrastructure, so an equally technical way to provide users a means (ideally a safe default) to resolve this should be made available and the people with skin in the game should chime in here.
it would be great if it were possible to separately define what algorithms are available by the provider to use, and what subset of those is advertised in TLS
That'd be a possible compromise indeed, but if memory serves,
openssl
now simply advertises all sig algs it finds providers to, well, provide. Admittedly, it was me adding this logic toopenssl
, so I feel at liberty to also change it again :-) @levitte: What'd be your take/recommendation in this regard? We could add a property "OSSL_CAPABILITY_TLS_SIGALG_ADVERTISE" or so to facilitate this.
Sounds like a complicated way for the provider to say "although I provide this, don't use it!"... unless I misunderstand the intention?
Not all the algorithms available in are allowed or welcome in TLS.
I'm looking for a way to make an algorithm usable in CMS and pkeyutl, but not cluttering/breaking TLS by its presence.
The only two alternatives so far are:
I find those alternatives worse. Do you concur?
Do you concur?
FWIW, I do: (PQ-)cert mgmt, CMS, etc all worked well before PQ-TLS-sig support. It'd be a pity to disable this by default just because of some weirdly behaving TLS servers.
there's a possibility that OpenSSL itself will get fixed
Is this so? Do you know for fact that the servers rejecting these "high-count sigalg code point" handshakes are running OpenSSL?
Is this so? Do you know for fact that the servers rejecting these "high-count sigalg code point" handshakes are running OpenSSL?
For sure we know that fwupd.org is not using OpenSSL. The TLS is terminated at the AWS load balancer, so s2n-tls is used in this case.
For sure we know that https://github.com/fwupd/fwupd/issues/7207#issuecomment-2095874862. The TLS is terminated at the AWS load balancer, so s2n-tls is used in this case.
Thanks for this confirmation: In this case, on OpenSSL fix (if necessary at all) would not buy us anything. Tagging @brian-jarvis-aws FYI (in case you have s2n contacts to chime in here).
Thanks, Brian brought this to s2n-tls's attention and we're currently working on a fix. It looks like we have a hard-coded limit of 64 signature schemes. There are <40 standardized signature schemes with assigned IANA values as of today, so having a max of 64 probably seemed like a safe assumption at the time. We are working to increase the limit as a quick short term fix, and longer term we plan to remove the limit completely.
there's a possibility that OpenSSL itself will get fixed
Is this so? Do you know for fact that the servers rejecting these "high-count sigalg code point" handshakes are running OpenSSL?
No I don't. I just thought that if the problem is with OpenSSL - then it may eventually get fixed.
For sure we know that fwupd.org is not TLS.
Fine. I've no clue what stack supports cargo.io. So, if we don't implement what I'm recommending - we'd need to either completely disable at least some OQS algorithms, or wait for indefinite time until all of those TLS stacks get fixed (if at all).
Thanks, Brian brought this to s2n-tls's attention and we're currently working on a fix. It looks like we have a hard-coded limit of 64 signature schemes. There are <40 standardized signature schemes with assigned IANA values as of today, so having a max of 64 probably seemed like a safe assumption at the time. We are working to increase the limit as a quick short term fix, and longer term we plan to remove the limit completely.
Cool that this will get fix, but then maybe the issue I observed with fwupd.org is something else, because the TLS handshake was failing with some scenarios with less than 64 sig algs (see this comment)
Cool that this will get fix, but then maybe the issue I observed with fwupd.org is something else, because the TLS handshake was failing with some scenarios with less than 64 sig algs (see https://github.com/open-quantum-safe/oqs-provider/issues/399#issuecomment-2085866168)
Are you sure that the complete setup sent less than 64 in this setup? If oqsprovider
"only" contributes 43 other providers may add some more pushing this above the AWS limit.
Fine. I've no clue what stack supports cargo.io. So, if we don't implement what I'm recommending - we'd need to either completely disable at least some OQS algorithms, or wait for indefinite time until all of those TLS stacks get fixed (if at all).
I'm not sure I get. Do you mean too many signature algorithms are advertised by the client? If yes, why don't you manage it on the system level?
too many signature algorithms are advertised by the client
That seems to be the problem. But then again, what is "too many" given there's no spec limiting this?
why don't you manage it on the system level?
How would you suggest doing this (short of creating a bespoke oqsprovider
version supporting (and reporting) fewer algs to openssl
)?
why don't you manage it on the system level?
How would you suggest doing this (short of creating a bespoke
oqsprovider
version supporting (and reporting) fewer algs toopenssl
)?
I mean smth like
[ ssl_module ]
system_default = crypto_policy
[ crypto_policy ]
...
SignatureAlgorithms = ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:ed25519:ed448:rsa_pss_pss_sha256:rsa_pss_pss_sha384:rsa_pss_pss_sha512:rsa_pss_rsae_sha256:rsa_pss_rsae_sha384:rsa_pss_rsae_sha512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224
Groups = X25519:secp256r1:X448:secp521r1:secp384r1:ffdhe2048:ffdhe3072:ffdhe4096:ffdhe6144:ffdhe8192
@beldmit and who (what app) is supposed to read policy file that you're proposing?
Thanks @beldmit ! The above seems to be the perfect solution not requiring any code changes; would you agree @mouse07410 ? Might this be a better way forward than https://github.com/open-quantum-safe/oqs-provider/pull/406, @iyanmv ?
@mouse07410 OpenSSL 3.x (otherwise providers are irrelevant) does it by default and it requires special efforts to cut this behavior off.
Are you talking about openssl.cnf
file? If so, and if somebody (other than myself) created the required entries - then yes I'd agree.
Otherwise - no.
Thanks @beldmit ! The above seems to be the perfect solution not requiring any code changes; would you agree @mouse07410 ? Might this be a better way forward than #406, @iyanmv ?
For sure! I like this solution.
Yes, I am talking about openssl.cnf
and don't see any reasons to not have such settings as a system default. You can easily override the default settings via passing a custom OpenSSL configuration file, if necessary.
Cool that this will get fix, but then maybe the issue I observed with fwupd.org is something else, because the TLS handshake was failing with some scenarios with less than 64 sig algs (see https://github.com/open-quantum-safe/oqs-provider/issues/399#issuecomment-2085866168)
Are you sure that the complete setup sent less than 64 in this setup? If oqsprovider "only" contributes 43 other providers may add some more pushing this above the AWS limit.
That's my guess as well. You could do a packet capture and check the ClientHello message signature_scheme extension to definitively confirm the number of signature schemes. But we were able to reproduce the failure with oqsprovider and verify that our fix resolves it-- you can see @goatgoose's testing in the description of https://github.com/aws/s2n-tls/pull/4544.
Cool that this will get fix, but then maybe the issue I observed with fwupd.org is something else, because the TLS handshake was failing with some scenarios with less than 64 sig algs (see #399 (comment))
Are you sure that the complete setup sent less than 64 in this setup? If oqsprovider "only" contributes 43 other providers may add some more pushing this above the AWS limit.
That's my guess as well. You could do a packet capture and check the ClientHello message signature_scheme extension to definitively confirm the number of signature schemes. But we were able to reproduce the failure with oqsprovider and verify that our fix resolves it-- you can see @goatgoose's testing in the description of aws/s2n-tls#4544.
Yes, yes. Sorry I forgot to answer to this before. I was indeed counting wrong the enabled sig algs (I think I was only counting those provided by oqsprovider). When I checked with wireshark, there were many more.
Yes, I am talking about
openssl.cnf
and don't see any reasons to not have such settings as a system default. You can easily override the default settings via passing a custom OpenSSL configuration file, if necessary.
However, the current openssl.cnf
does not include such settings. So, somebody has to add that support to OpenSSL, and then backport it to the stable release(s).
How would it deal with algorithms from providers that are disabled at the moment (for whatever reason)?
On top of that, I'm not crazy about writing down all the algorithms to the tune of ...:rsa_pss_rsae_sha384:rsa_pss_rsae_sha512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:...
from all the providers, that I want supported system-wide. Who's going to do that? @beldmit you, perhaps? @baentsch maybe you? Or will OpenSSL maintainers just add the keyword default
that would include all the algorithms from all the enabled providers, and we'll be back at square one?
Which is why being able to specify for a provider what algorithms it should advertise to TLS, seems easier in the short term.
I was indeed counting wrong the enabled sig algs (I think I was only counting those provided by oqsprovider). When I checked with wireshark, there were many more.
That merely means that the actual limit that was exceeded when 60+ algorithms from oqsprovider were added, was larger than 64.
Before answering I kindly ask you to make your tone less personal.
Yes, I am talking about
openssl.cnf
and don't see any reasons to not have such settings as a system default. You can easily override the default settings via passing a custom OpenSSL configuration file, if necessary.However, the current
openssl.cnf
does not include such settings. So, somebody has to add that support to OpenSSL, and then backport it to the stable release(s).
openssl.cnf
is a configuration file. It means it can be tuned on each and every system (or left unchanged if it's not necessary). OpenSSL supports and documents this configuration for ages.
How would it deal with algorithms from providers that are disabled at the moment (for whatever reason)? On top of that, I'm not crazy about writing down all the algorithms to the tune of
...:rsa_pss_rsae_sha384:rsa_pss_rsae_sha512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:...
from all the providers, that I want supported system-wide. Who's going to do that? @beldmit you, perhaps? @baentsch maybe you? Or will OpenSSL maintainers just add the keyworddefault
that would include all the algorithms from all the enabled providers, and we'll be back at square one?
It should be done by the same person who installed and activated the particular provider, right?
If a particular setting (e.g. SignatureAlgorithms) is omitted, then no limits are applied so there is no need to provide the default
keyword.
Which is why being able to specify for a provider what algorithms it should advertise to TLS, seems easier in the short term.
Let me disagree.
Perhaps this section in the USAGE.md could be expanded a little bit to satisfy @mouse07410 with a warning about that some TLS servers may fail if too many sig algs are offered by the client and how to avoid the problem with a "safe" (short enough) selection of signature algorithms as an example (like in this comment but perhaps using the same Groups from https://github.com/open-quantum-safe/oqs-demos/blob/main/nginx/fulltest/Dockerfile#L25-L26).
I think the main problem for @mouse07410 (please, correct me if I understood you wrong) is that, at the moment, some servers fail when oqsprovider is used with the default enabled sig algs and with the default OpenSSL config file.
Perhaps this section in the USAGE.md could be expanded a little bit
That's precisely the quick "fix" I also had in mind. I just wanted to experiment a bit with different settings to propose something tested/validated against these "bad" servers -- and maybe already add that into a recommended default "openssl.cnf" (can't be the test one as we then could not test all sigalgs in TLS any more). Anyone beating me to a PR along those lines very welcome :)
Before answering I kindly ask you to make your tone less personal.
Yes, of course. My apologies.
openssl.cnf
is a configuration file. It means it can be tuned on each and every system (or left unchanged if it's not necessary). OpenSSL supports and documents this configuration for ages.
Of course. But first, somebody would have to add what you propose to the code that deals with this file, and I suspect it's more work than you're portraying.
Second, and more importantly - it will require much more knowledge of the normal user of the "guts" of the OpenSSL than IMHO is reasonable to expect.
For example, I have no clue what signature (or KEM, or whatever) algorithms OpenSSL supports through all of its providers, and even less so - what algorithms it exports via TLS.
I don't think it's fair to expect a "normal" user - even if/when this .cnf
feature gets suported! - to list all the algorithms he needs "his" TLS to support, merely because one provider offers a few extra signature algorithms that "break the back of the camel".
Perhaps this section in the USAGE.md could be expanded a little bit to satisfy @mouse07410 with a warning about that some TLS servers may fail if too many sig algs are offered by the client and how to avoid the problem with a "safe" (short enough) selection of signature algorithms as an example (like in https://github.com/open-quantum-safe/oqs-provider/issues/399#issuecomment-2097815008
No. I don't need this warning - I need to know what exactly to put into openssl.cnf
file so that (a) I can use all the algorithms that all of the providers offer within something like pkeyutl
, but (b) exclude from TLS some algorithms that I know oqsprovider offers.
And I asked (perhaps, in an impolite way) who is going to provide that text suitable for direct includion into openssl.cnf
. Surely, you don't expect a user to be knowledgable enough to do that?!
That's precisely the quick "fix" I also had in mind.
I'm pretty sure this "fix" will not work for me. Thankfully, there's generate.yml
file. I guess I can live with several non-NIST PQ signature algorithms omitted from crypto
.
Surely, you don't expect a user to be knowledgable enough to do that?!
Surely not. I tried to state that I'll give it a go but stupidly got lost doing that as my "Linux Foundation allergy" on other issues sapped all my energy. Getting too old. Sorry.
somebody would have to add what you propose to the code that deals with this file, and I suspect it's more work than you're portraying.
My idea was to add code to the oqsprovider
code generator targeting (generating) a "recommended default" openssl.cnf
listing 2 or 3 classic algs and then all plain NIST PQ std algs (plus some documentation around that, of course). But doing that with sufficient automation that it survives future external contributions indeed needs some more work.
OK, so here are the alternatives that I see:
README.md
, suggesting that if a user needs to TLS-communicate with a "sensitive" server, she should edit generate.yml
file, disabling signatures she doesn't need, and rebuilding oqs-provider
. This will completely remove disabled sigs (neither TLS nor pkeyutl
or cms
would have access to them).[ssl-signatures]
option in openssl.cnf
, and add examples to README.md
or USAGE.md
, giving the exact strings the user would put in that openssl.cnf
field to achieve the goal of having all sigs available in cms
and fewer in TLS. As other providers change and/or get added, the values of those example strings would need to change to keep up.OSSL_CAPABILITY_TLS_SIGALG_ADVERTISE
config flag to the oqs-provider
. The only dependency then would be internal to this provider, all the PQ signature algorithms would be available to commands like cms
, and "sensitive" servers would be OK with ClientHello received from this installation.Needless to say, I prefer (3), while currently using (1). I don't think (2) is sustainable in the long run, unless the community agrees to (a) add such support, and (b) maintain such strings.
[2] is supported in OpenSSL so I strongly prefer this solution. Installing oqsprovider is not a common configuration and anyway implies changing the openssl.cnf
file for activation
[2] is supported in OpenSSL so I strongly prefer this solution.
Are you saying that [ssl-signatures]
parameter in openssl.cnf
is already supported? If so, references, please? Ideally - both to the code that implements it and to the man
page that describes it?
Installing oqsprovider is not a common configuration and anyway implies changing the
openssl.cnf
file for activation
Yes. But even an otherwise-ignorant user like yours-truly can add a few trivial lines to openssl.cnf
that enable oqsprovider. And the process is both short enough and well-documented.
While listing all the acceptable signature algorithms, especially when you factor in the providers, is IMHO an insurmountable task - and the one that no one can expect normal users to perform.
So, unless there are "cribs", aka - examples of complete parameter-strings that would preserve the current TLS behavior with the Classic algorithms and give "sub-strings" to append for adding algorithms from providers (probably, for every provider) - I'm strongly against the [2], because it is the best in theory but very much unusable in practice.
https://openssl.org/docs/man3.2/man5/config.html, SSL Configuration https://openssl.org/docs/man3.2/man3/SSL_CONF_cmd.html
It is here since 1.1.1
But I totally agree that the provider-specific examples of these commands are necessary
https://openssl.org/docs/man3.2/man5/config.html, SSL Configuration
Sorry, I must be dense today - but I did not find anything there that lists signatures algorithms that should be allowed/enabled for TLS. :-(
Quoting:
Each configuration section consists of name/value pairs that are parsed by SSL_CONF_cmd(3), which will be called by SSL_CTX_config() or SSL_config(), appropriately.
@beldmit thanks, but I still don't understand. I cannot find field sig-signatures
, or a similarly-named field anywhere in the OpenSSL codebase (master branch). I noticed the flag -sigalgs
apparently available in openssl s_client
command-line, but I don't see how it translates to availability in openssl.cnf
. Would you mind showing exactly how to add entries to the OpenSSL config file now? And if you know - where to get the complete list of the supported algorithms in a format suitable for inclusion in the config file?
IMHO, yet another example of stuff not being ready for a "normal" user. Because if to accomplish even such a simple step one has to ask experts - it's not ready by my book.
It's named SignatureAlgorithms. And yes, any documentation contribution to OpenSSL is welcome
It's named SignatureAlgorithms. And yes, any documentation contribution to OpenSSL is welcome
It makes sense that those who understand the subject contribute documentation. E.g., what would you expect me to contrbute - my questions to you?
Sorry, let me disagree. I'm aware of the structure of the documentation and know where to look for it. If you find not obvious the entry point to the documentation, please let me know what you think to be a valid starting point to look.
If you find not obvious the entry point to the documentation, please let me know what you think to be a valid starting point to look.
Oh sure! Where do I begin? :-)
First, an obvious place for me to learn about the contents of the config file would be (unsurprisingly!) something like man openssl-config
or man config
. OK, present. I proceed down the page and find [ssl_configuration]
. OK, excellent. It further points me at [client_tls_config]
and [server_tls_config]
. Great - but what are the values that I can put there? Here I stumble. Either there should be a list of all of the supported keys/parameter-names/whatever, or a reference like "to see the list of all the keys do man config-tls
". And then - for each key needs to be a list of all the possible/legal values, and/or a way to get it (in the correct/required syntax).
How's that for the beginning? ;-)
So the config
manual page cuurently has the wording
Each configuration section consists of name/value pairs that are parsed by SSL_CONF_cmd(3)
What's wrong here and could you please submit a PR to make it better?
What's wrong here?
The mere fact that neither a list (let alone a complete list) of the keys and their allowed values is given, nor is the format (suitable for inclusion in openssl.cnf
) of those mentioned in SSL_CONF_cmd
specified. Also, as I said before - lack of the list of all the algorithms in format suitable for direct inclusion in openssl.cnf
.
As I also already said, it's impractical to expect a normal user to know those.
could you please submit a PR to make it better?
If I knew the answers to the above questions - I wouldn't bother asking people here and spending my time on this discussion. Instead, I would've just edited openssl.cnf
and got done with it. Unfortunately, I don't have those answers - so, here I am. :-(
What's wrong here and could you please submit a PR to make it better?
First step done in https://github.com/openssl/openssl/pull/24499
@mouse07410 The list of default-active sigalgs is indeed documented in SSL_CONF_cmd(3), though not in a way that it can be written down immediately. Also, it's dependent on the activated providers (as clarified in the PR above just created) which is tedious but a consequence of the dynamic provider concept.
@beldmit Are you aware of a facility within openssl
to extract all such (currently registered) algorithm combinations along the lines (but extending to all permitted configurable sigalg combinations) of openssl list -signature-algorithms
?
And another question if I may, @beldmit : Do you know where the semantics of "..." is documented as you used it in your "crypto_policy" example above? When "playing around" with a .cnf file now generated as part of an enhanced oqsprovider
generator script, these three dots make a massive difference as to what works and what doesn't.
@baentsch In Fedora we ship oqsprovider/liboqs and have the corresponding PQ policy. Currently it provides all the algorithms supported by liboqs
@baentsch In Fedora we ship oqsprovider/liboqs and have the corresponding PQ policy.
Thanks for letting me know, but my question pertains to openssl
(config options in general). Should we create a separate discussion item there?
Currently it provides all the algorithms supported by liboqs
So does this mean you disable there all hybrid and composite algs, thus working around this issue by supporting fewer than 64 sigalgs?
The list of default-active sigalgs is indeed documented in SSL_CONF_cmd(3), though not in a way that it can be written down immediately.
Which is one of my points - a user would have to figure out how to write them down in a proper form. I don't think such an expectation is reasonable.
Also, it's dependent on the activated providers (as clarified in the PR above just created) which is tedious but a consequence of the dynamic provider concept.
Which is my other point - we can't get away from dynamic providers (nor do we want to), so the process becomes not merely tedious, but requiring an expert.
I say again - this is a nice theoretic solution that has no room in the ugly practical world.
Describe the bug Provider built from the
main
branch pulled after Fri Apr 12, 2024, somehow causes OpenSSL to hang and then time-out on requests over corporate firewall (to https://index.crates.io, in case it matters).When I comment out
oqs
provider inopenssl.cnf
the problem disappears.I must add that before Apr 12th everything worked just fine. So, it's OpenSSL, or liboqs, or oqs-provider.
@levitte could you please take a look as well? I don't know whether it's the provider's fault, or that of the OpenSSL itself.
To Reproduce A little complicated, but here's what I have.
Steps to reproduce the behavior:
cargo-update
viacargo install cargo-update
liboqs
.oqs-provider
(main
branch).openssl.cnf
to addoqs
provider (some add it asoqsprovider
, for me naming itoqs
suffices).cargo install-update -l
Expected behavior
Something like
Actual behavior
Environment (please complete the following information):
main
is)Please run the following commands to obtain the version information:
openssl version
openssl list -providers