search

open-quantum-safe / oqs-provider

OpenSSL 3 provider containing post-quantum algorithms
https://openquantumsafe.org
MIT License
192 stars 80 forks source link

Too many advertised sig algs cause TLS server hang-up #399

Open mouse07410 opened 4 months ago

mouse07410 commented 4 months ago

Describe the bug Provider built from the main branch pulled after Fri Apr 12, 2024, somehow causes OpenSSL to hang and then time-out on requests over corporate firewall (to https://index.crates.io, in case it matters).

When I comment out oqs provider in openssl.cnf the problem disappears.

I must add that before Apr 12th everything worked just fine. So, it's OpenSSL, or liboqs, or oqs-provider.

@levitte could you please take a look as well? I don't know whether it's the provider's fault, or that of the OpenSSL itself.

To Reproduce A little complicated, but here's what I have.

Steps to reproduce the behavior:

  1. Install Rust toolchain.
  2. Install cargo-update via cargo install cargo-update
  3. Have OpenSSL-3.2.1 installed.
  4. Install current master of liboqs.
  5. Clone and install oqs-provider (main branch).
  6. Edit openssl.cnf to add oqs provider (some add it as oqsprovider, for me naming it oqs suffices).
  7. Try cargo install-update -l
  8. See error

Expected behavior

Something like

$ cargo install-update -l
    Polling registry 'https://index.crates.io/'.......................................

Package          Installed             Latest                               Needs update
asn1rs           v0.3.1                v0.3.1                               No
b3sum            v1.5.1                v1.5.1                               No
.  .  .

Actual behavior

$ cargo install-update -l
    Polling registry 'https://index.crates.io/'
Failed to update index repository crates-io: package asn1rs: [35] SSL connect error (OpenSSL SSL_connect: SSL_ERROR_ZERO_RETURN in connection to index.crates.io:443 ).
$ 
$ OQSPROV=1 cargo install-update -l
OQS PROV: successfully registered dilithium2 with NID 1320
OQS PROV: successfully registered p256_dilithium2 with NID 1321
OQS PROV: successfully registered rsa3072_dilithium2 with NID 1322
OQS PROV: successfully registered dilithium3 with NID 1323
OQS PROV: successfully registered p384_dilithium3 with NID 1324
OQS PROV: successfully registered dilithium5 with NID 1325
OQS PROV: successfully registered p521_dilithium5 with NID 1326
OQS PROV: successfully registered mldsa44 with NID 1327
OQS PROV: successfully registered p256_mldsa44 with NID 1328
OQS PROV: successfully registered rsa3072_mldsa44 with NID 1329
OQS PROV: successfully registered mldsa44_pss2048 with NID 1330
OQS PROV: successfully registered mldsa44_rsa2048 with NID 1331
OQS PROV: successfully registered mldsa44_ed25519 with NID 1332
OQS PROV: successfully registered mldsa44_p256 with NID 1333
OQS PROV: successfully registered mldsa44_bp256 with NID 1334
OQS PROV: successfully registered mldsa65 with NID 1335
OQS PROV: successfully registered p384_mldsa65 with NID 1336
OQS PROV: successfully registered mldsa65_pss3072 with NID 1337
OQS PROV: successfully registered mldsa65_rsa3072 with NID 1338
OQS PROV: successfully registered mldsa65_p256 with NID 1339
OQS PROV: successfully registered mldsa65_bp256 with NID 1340
OQS PROV: successfully registered mldsa65_ed25519 with NID 1341
OQS PROV: successfully registered mldsa87 with NID 1342
OQS PROV: successfully registered p521_mldsa87 with NID 1343
OQS PROV: successfully registered mldsa87_p384 with NID 1344
OQS PROV: successfully registered mldsa87_bp384 with NID 1345
OQS PROV: successfully registered mldsa87_ed448 with NID 1346
OQS PROV: successfully registered falcon512 with NID 1347
OQS PROV: successfully registered p256_falcon512 with NID 1348
OQS PROV: successfully registered rsa3072_falcon512 with NID 1349
OQS PROV: successfully registered falconpadded512 with NID 1350
OQS PROV: successfully registered p256_falconpadded512 with NID 1351
OQS PROV: successfully registered rsa3072_falconpadded512 with NID 1352
OQS PROV: successfully registered falcon1024 with NID 1353
OQS PROV: successfully registered p521_falcon1024 with NID 1354
OQS PROV: successfully registered falconpadded1024 with NID 1355
OQS PROV: successfully registered p521_falconpadded1024 with NID 1356
OQS PROV: successfully registered sphincssha2128fsimple with NID 1357
OQS PROV: successfully registered p256_sphincssha2128fsimple with NID 1358
OQS PROV: successfully registered rsa3072_sphincssha2128fsimple with NID 1359
OQS PROV: successfully registered sphincssha2128ssimple with NID 1360
OQS PROV: successfully registered p256_sphincssha2128ssimple with NID 1361
OQS PROV: successfully registered rsa3072_sphincssha2128ssimple with NID 1362
OQS PROV: successfully registered sphincssha2192fsimple with NID 1363
OQS PROV: successfully registered p384_sphincssha2192fsimple with NID 1364
OQS PROV: successfully registered sphincsshake128fsimple with NID 1365
OQS PROV: successfully registered p256_sphincsshake128fsimple with NID 1366
OQS PROV: successfully registered rsa3072_sphincsshake128fsimple with NID 1367
OQS PROV: Default or FIPS provider available.
    Polling registry 'https://index.crates.io/'Unknown operation 5 requested from OQS provider
Unknown operation 5 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 2 requested from OQS provider
Unknown operation 1 requested from OQS provider
Unknown operation 1 requested from OQS provider
Unknown operation 1 requested from OQS provider
Unknown operation 1 requested from OQS provider
Unknown operation 1 requested from OQS provider
Unknown operation 1 requested from OQS provider
Unknown operation 1 requested from OQS provider
Unknown operation 11 requested from OQS provider
Unknown operation 11 requested from OQS provider

Failed to update index repository crates-io: package asn1rs: [35] SSL connect error (OpenSSL SSL_connect: SSL_ERROR_ZERO_RETURN in connection to index.crates.io:443 ).
$

Environment (please complete the following information):

Please run the following commands to obtain the version information:

$ openssl version
OpenSSL 3.2.1 30 Jan 2024 (Library: OpenSSL 3.2.1 30 Jan 2024)
$ openssl list -providers
Providers:
  base
    name: OpenSSL Base Provider
    version: 3.2.1
    status: active
  default
    name: OpenSSL Default Provider
    version: 3.2.1
    status: active
  legacy
    name: OpenSSL Legacy Provider
    version: 3.2.1
    status: active
  oqs
    name: OpenSSL OQS Provider
    version: 0.6.0
    status: active
  pkcs11
    name: PKCS#11 Provider
    version: 3.2.1
    status: active
$
baentsch commented 4 months ago

@baentsch I strongly suggest changing the defaults (at least for now) in oqs-templates/generate.yml, and disabling all the signature algorithms except for ML-DSA.

That'd be pretty drastic, no? Would you suggest also disabling the hybrid and composite variants, @mouse07410 ? @beldmit: What'd be your position: You seem to have taken to using some of the (now) "historic" variants of PQCrystals algs: Would you also be OK if we'd disable them by default? Any objections, @dstebila ? Researchers can always re-enable algs they need, but normal users should not be negatively impacted by shenanigans like these, so I currently tend to support this proposal.

beldmit commented 4 months ago

I am going to enable "old" Kyber for Fedora 41 (rawhide) at least for now for interoperability's sake for liboqs and oqsprovider

dstebila commented 3 months ago

@baentsch I strongly suggest changing the defaults (at least for now) in oqs-templates/generate.yml, and disabling all the signature algorithms except for ML-DSA.

That'd be pretty drastic, no? Would you suggest also disabling the hybrid and composite variants, @mouse07410 ? @beldmit: What'd be your position: You seem to have taken to using some of the (now) "historic" variants of PQCrystals algs: Would you also be OK if we'd disable them by default? Any objections, @dstebila ? Researchers can always re-enable algs they need, but normal users should not be negatively impacted by shenanigans like these, so I currently tend to support this proposal.

I think it would be a bit drastic to disable everything except ML-DSA. I'm not sure what the right combination is, but hopefully we can get a reasonable mix in. This is where an "adopter advisory committee" would be helpful...

mouse07410 commented 3 months ago

That'd be pretty drastic, no?

Given that the only working alternative seems to be disabling the oqs-provider altogether - probably not that drastic. Of course, I'm all ears to hear what options you can recommend.

Situation:

What currently works: I've successfully tested the following (using modifications proposed by @iyanmv above):

Would you suggest also disabling the hybrid and composite variants, @mouse07410 ?

The main reason for disabling some of the algorithms/variants is to make TLS work without removing oqs-provider altogether. Thus, if (as it appears to me) hybrid and composite variants aren't automatically disabled by modifications defined here - then no. If they are disabled - then yes.

I'm not sure what the right combination is, but hopefully we can get a reasonable mix in.

Offhand and IMHO, whatever from the NIST standards that fits TLS well. So far, that's Falcon and ML-DSA.

Update

I don't know the scope of enabling/disabling algorithms. IMHO, it would be great if it were possible to separately define what algorithms are available by the provider to use, and what subset of those is advertised in TLS. In that case, I'd have everything enabled in the provider, but only the NIST variants (except SPHINCS+) advertised in TLS ClientHello.

baentsch commented 3 months ago

it would be great if it were possible to separately define what algorithms are available by the provider to use, and what subset of those is advertised in TLS

That'd be a possible compromise indeed, but if memory serves, openssl now simply advertises all sig algs it finds providers to, well, provide. Admittedly, it was me adding this logic to openssl, so I feel at liberty to also change it again :-) @levitte: What'd be your take/recommendation in this regard? We could add a property "OSSL_CAPABILITY_TLS_SIGALG_ADVERTISE" or so to facilitate this.

baentsch commented 3 months ago

This is where an "adopter advisory committee" would be helpful...

I'm not sure another committee would help here: This simply seems to be a technical limitation by existing infrastructure, so an equally technical way to provide users a means (ideally a safe default) to resolve this should be made available and the people with skin in the game should chime in here.

levitte commented 3 months ago

it would be great if it were possible to separately define what algorithms are available by the provider to use, and what subset of those is advertised in TLS

That'd be a possible compromise indeed, but if memory serves, openssl now simply advertises all sig algs it finds providers to, well, provide. Admittedly, it was me adding this logic to openssl, so I feel at liberty to also change it again :-) @levitte: What'd be your take/recommendation in this regard? We could add a property "OSSL_CAPABILITY_TLS_SIGALG_ADVERTISE" or so to facilitate this.

Sounds like a complicated way for the provider to say "although I provide this, don't use it!"... unless I misunderstand the intention?

mouse07410 commented 3 months ago

Not all the algorithms available in are allowed or welcome in TLS.

I'm looking for a way to make an algorithm usable in CMS and pkeyutl, but not cluttering/breaking TLS by its presence.

The only two alternatives so far are:

I find those alternatives worse. Do you concur?

baentsch commented 3 months ago

Do you concur?

FWIW, I do: (PQ-)cert mgmt, CMS, etc all worked well before PQ-TLS-sig support. It'd be a pity to disable this by default just because of some weirdly behaving TLS servers.

there's a possibility that OpenSSL itself will get fixed

Is this so? Do you know for fact that the servers rejecting these "high-count sigalg code point" handshakes are running OpenSSL?

iyanmv commented 3 months ago

Is this so? Do you know for fact that the servers rejecting these "high-count sigalg code point" handshakes are running OpenSSL?

For sure we know that fwupd.org is not using OpenSSL. The TLS is terminated at the AWS load balancer, so s2n-tls is used in this case.

baentsch commented 3 months ago

For sure we know that https://github.com/fwupd/fwupd/issues/7207#issuecomment-2095874862. The TLS is terminated at the AWS load balancer, so s2n-tls is used in this case.

Thanks for this confirmation: In this case, on OpenSSL fix (if necessary at all) would not buy us anything. Tagging @brian-jarvis-aws FYI (in case you have s2n contacts to chime in here).

lrstewart commented 3 months ago

Thanks, Brian brought this to s2n-tls's attention and we're currently working on a fix. It looks like we have a hard-coded limit of 64 signature schemes. There are <40 standardized signature schemes with assigned IANA values as of today, so having a max of 64 probably seemed like a safe assumption at the time. We are working to increase the limit as a quick short term fix, and longer term we plan to remove the limit completely.

mouse07410 commented 3 months ago

there's a possibility that OpenSSL itself will get fixed

Is this so? Do you know for fact that the servers rejecting these "high-count sigalg code point" handshakes are running OpenSSL?

No I don't. I just thought that if the problem is with OpenSSL - then it may eventually get fixed.

For sure we know that fwupd.org is not TLS.

Fine. I've no clue what stack supports cargo.io. So, if we don't implement what I'm recommending - we'd need to either completely disable at least some OQS algorithms, or wait for indefinite time until all of those TLS stacks get fixed (if at all).

iyanmv commented 3 months ago

Thanks, Brian brought this to s2n-tls's attention and we're currently working on a fix. It looks like we have a hard-coded limit of 64 signature schemes. There are <40 standardized signature schemes with assigned IANA values as of today, so having a max of 64 probably seemed like a safe assumption at the time. We are working to increase the limit as a quick short term fix, and longer term we plan to remove the limit completely.

Cool that this will get fix, but then maybe the issue I observed with fwupd.org is something else, because the TLS handshake was failing with some scenarios with less than 64 sig algs (see this comment)

baentsch commented 3 months ago

Cool that this will get fix, but then maybe the issue I observed with fwupd.org is something else, because the TLS handshake was failing with some scenarios with less than 64 sig algs (see https://github.com/open-quantum-safe/oqs-provider/issues/399#issuecomment-2085866168)

Are you sure that the complete setup sent less than 64 in this setup? If oqsprovider "only" contributes 43 other providers may add some more pushing this above the AWS limit.

beldmit commented 3 months ago

Fine. I've no clue what stack supports cargo.io. So, if we don't implement what I'm recommending - we'd need to either completely disable at least some OQS algorithms, or wait for indefinite time until all of those TLS stacks get fixed (if at all).

I'm not sure I get. Do you mean too many signature algorithms are advertised by the client? If yes, why don't you manage it on the system level?

baentsch commented 3 months ago

too many signature algorithms are advertised by the client

That seems to be the problem. But then again, what is "too many" given there's no spec limiting this?

why don't you manage it on the system level?

How would you suggest doing this (short of creating a bespoke oqsprovider version supporting (and reporting) fewer algs to openssl)?

beldmit commented 3 months ago

why don't you manage it on the system level?

How would you suggest doing this (short of creating a bespoke oqsprovider version supporting (and reporting) fewer algs to openssl)?

I mean smth like

[ ssl_module ]  
system_default = crypto_policy

[ crypto_policy ]
...
SignatureAlgorithms = ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:ed25519:ed448:rsa_pss_pss_sha256:rsa_pss_pss_sha384:rsa_pss_pss_sha512:rsa_pss_rsae_sha256:rsa_pss_rsae_sha384:rsa_pss_rsae_sha512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224
Groups = X25519:secp256r1:X448:secp521r1:secp384r1:ffdhe2048:ffdhe3072:ffdhe4096:ffdhe6144:ffdhe8192
mouse07410 commented 3 months ago

@beldmit and who (what app) is supposed to read policy file that you're proposing?

baentsch commented 3 months ago

Thanks @beldmit ! The above seems to be the perfect solution not requiring any code changes; would you agree @mouse07410 ? Might this be a better way forward than https://github.com/open-quantum-safe/oqs-provider/pull/406, @iyanmv ?

beldmit commented 3 months ago

@mouse07410 OpenSSL 3.x (otherwise providers are irrelevant) does it by default and it requires special efforts to cut this behavior off.

mouse07410 commented 3 months ago

Are you talking about openssl.cnf file? If so, and if somebody (other than myself) created the required entries - then yes I'd agree.

Otherwise - no.

iyanmv commented 3 months ago

Thanks @beldmit ! The above seems to be the perfect solution not requiring any code changes; would you agree @mouse07410 ? Might this be a better way forward than #406, @iyanmv ?

For sure! I like this solution.

beldmit commented 3 months ago

Yes, I am talking about openssl.cnf and don't see any reasons to not have such settings as a system default. You can easily override the default settings via passing a custom OpenSSL configuration file, if necessary.

lrstewart commented 3 months ago

Cool that this will get fix, but then maybe the issue I observed with fwupd.org is something else, because the TLS handshake was failing with some scenarios with less than 64 sig algs (see https://github.com/open-quantum-safe/oqs-provider/issues/399#issuecomment-2085866168)

Are you sure that the complete setup sent less than 64 in this setup? If oqsprovider "only" contributes 43 other providers may add some more pushing this above the AWS limit.

That's my guess as well. You could do a packet capture and check the ClientHello message signature_scheme extension to definitively confirm the number of signature schemes. But we were able to reproduce the failure with oqsprovider and verify that our fix resolves it-- you can see @goatgoose's testing in the description of https://github.com/aws/s2n-tls/pull/4544.

iyanmv commented 3 months ago

Cool that this will get fix, but then maybe the issue I observed with fwupd.org is something else, because the TLS handshake was failing with some scenarios with less than 64 sig algs (see #399 (comment))

Are you sure that the complete setup sent less than 64 in this setup? If oqsprovider "only" contributes 43 other providers may add some more pushing this above the AWS limit.

That's my guess as well. You could do a packet capture and check the ClientHello message signature_scheme extension to definitively confirm the number of signature schemes. But we were able to reproduce the failure with oqsprovider and verify that our fix resolves it-- you can see @goatgoose's testing in the description of aws/s2n-tls#4544.

Yes, yes. Sorry I forgot to answer to this before. I was indeed counting wrong the enabled sig algs (I think I was only counting those provided by oqsprovider). When I checked with wireshark, there were many more.

mouse07410 commented 3 months ago

Yes, I am talking about openssl.cnf and don't see any reasons to not have such settings as a system default. You can easily override the default settings via passing a custom OpenSSL configuration file, if necessary.

However, the current openssl.cnf does not include such settings. So, somebody has to add that support to OpenSSL, and then backport it to the stable release(s).

How would it deal with algorithms from providers that are disabled at the moment (for whatever reason)? On top of that, I'm not crazy about writing down all the algorithms to the tune of ...:rsa_pss_rsae_sha384:rsa_pss_rsae_sha512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:... from all the providers, that I want supported system-wide. Who's going to do that? @beldmit you, perhaps? @baentsch maybe you? Or will OpenSSL maintainers just add the keyword default that would include all the algorithms from all the enabled providers, and we'll be back at square one?

Which is why being able to specify for a provider what algorithms it should advertise to TLS, seems easier in the short term.

I was indeed counting wrong the enabled sig algs (I think I was only counting those provided by oqsprovider). When I checked with wireshark, there were many more.

That merely means that the actual limit that was exceeded when 60+ algorithms from oqsprovider were added, was larger than 64.

beldmit commented 3 months ago

Before answering I kindly ask you to make your tone less personal.

Yes, I am talking about openssl.cnf and don't see any reasons to not have such settings as a system default. You can easily override the default settings via passing a custom OpenSSL configuration file, if necessary.

However, the current openssl.cnf does not include such settings. So, somebody has to add that support to OpenSSL, and then backport it to the stable release(s).

openssl.cnf is a configuration file. It means it can be tuned on each and every system (or left unchanged if it's not necessary). OpenSSL supports and documents this configuration for ages.

How would it deal with algorithms from providers that are disabled at the moment (for whatever reason)? On top of that, I'm not crazy about writing down all the algorithms to the tune of ...:rsa_pss_rsae_sha384:rsa_pss_rsae_sha512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:... from all the providers, that I want supported system-wide. Who's going to do that? @beldmit you, perhaps? @baentsch maybe you? Or will OpenSSL maintainers just add the keyword default that would include all the algorithms from all the enabled providers, and we'll be back at square one?

It should be done by the same person who installed and activated the particular provider, right? If a particular setting (e.g. SignatureAlgorithms) is omitted, then no limits are applied so there is no need to provide the default keyword.

Which is why being able to specify for a provider what algorithms it should advertise to TLS, seems easier in the short term.

Let me disagree.

iyanmv commented 3 months ago

Perhaps this section in the USAGE.md could be expanded a little bit to satisfy @mouse07410 with a warning about that some TLS servers may fail if too many sig algs are offered by the client and how to avoid the problem with a "safe" (short enough) selection of signature algorithms as an example (like in this comment but perhaps using the same Groups from https://github.com/open-quantum-safe/oqs-demos/blob/main/nginx/fulltest/Dockerfile#L25-L26).

I think the main problem for @mouse07410 (please, correct me if I understood you wrong) is that, at the moment, some servers fail when oqsprovider is used with the default enabled sig algs and with the default OpenSSL config file.

baentsch commented 3 months ago

Perhaps this section in the USAGE.md could be expanded a little bit

That's precisely the quick "fix" I also had in mind. I just wanted to experiment a bit with different settings to propose something tested/validated against these "bad" servers -- and maybe already add that into a recommended default "openssl.cnf" (can't be the test one as we then could not test all sigalgs in TLS any more). Anyone beating me to a PR along those lines very welcome :)

mouse07410 commented 3 months ago

Before answering I kindly ask you to make your tone less personal.

Yes, of course. My apologies.

openssl.cnf is a configuration file. It means it can be tuned on each and every system (or left unchanged if it's not necessary). OpenSSL supports and documents this configuration for ages.

Of course. But first, somebody would have to add what you propose to the code that deals with this file, and I suspect it's more work than you're portraying.

Second, and more importantly - it will require much more knowledge of the normal user of the "guts" of the OpenSSL than IMHO is reasonable to expect.

For example, I have no clue what signature (or KEM, or whatever) algorithms OpenSSL supports through all of its providers, and even less so - what algorithms it exports via TLS.

I don't think it's fair to expect a "normal" user - even if/when this .cnf feature gets suported! - to list all the algorithms he needs "his" TLS to support, merely because one provider offers a few extra signature algorithms that "break the back of the camel".

Perhaps this section in the USAGE.md could be expanded a little bit to satisfy @mouse07410 with a warning about that some TLS servers may fail if too many sig algs are offered by the client and how to avoid the problem with a "safe" (short enough) selection of signature algorithms as an example (like in https://github.com/open-quantum-safe/oqs-provider/issues/399#issuecomment-2097815008

No. I don't need this warning - I need to know what exactly to put into openssl.cnf file so that (a) I can use all the algorithms that all of the providers offer within something like pkeyutl, but (b) exclude from TLS some algorithms that I know oqsprovider offers.

And I asked (perhaps, in an impolite way) who is going to provide that text suitable for direct includion into openssl.cnf. Surely, you don't expect a user to be knowledgable enough to do that?!

That's precisely the quick "fix" I also had in mind.

I'm pretty sure this "fix" will not work for me. Thankfully, there's generate.yml file. I guess I can live with several non-NIST PQ signature algorithms omitted from crypto.

baentsch commented 3 months ago

Surely, you don't expect a user to be knowledgable enough to do that?!

Surely not. I tried to state that I'll give it a go but stupidly got lost doing that as my "Linux Foundation allergy" on other issues sapped all my energy. Getting too old. Sorry.

somebody would have to add what you propose to the code that deals with this file, and I suspect it's more work than you're portraying.

My idea was to add code to the oqsprovider code generator targeting (generating) a "recommended default" openssl.cnf listing 2 or 3 classic algs and then all plain NIST PQ std algs (plus some documentation around that, of course). But doing that with sufficient automation that it survives future external contributions indeed needs some more work.

mouse07410 commented 3 months ago

OK, so here are the alternatives that I see:

  1. Do nothing, except for maybe documenting the problem in the README.md, suggesting that if a user needs to TLS-communicate with a "sensitive" server, she should edit generate.yml file, disabling signatures she doesn't need, and rebuilding oqs-provider. This will completely remove disabled sigs (neither TLS nor pkeyutl or cms would have access to them).
  2. Get OpenSSL to support/implement [ssl-signatures] option in openssl.cnf, and add examples to README.md or USAGE.md, giving the exact strings the user would put in that openssl.cnf field to achieve the goal of having all sigs available in cms and fewer in TLS. As other providers change and/or get added, the values of those example strings would need to change to keep up.
  3. Add OSSL_CAPABILITY_TLS_SIGALG_ADVERTISE config flag to the oqs-provider. The only dependency then would be internal to this provider, all the PQ signature algorithms would be available to commands like cms, and "sensitive" servers would be OK with ClientHello received from this installation.

Needless to say, I prefer (3), while currently using (1). I don't think (2) is sustainable in the long run, unless the community agrees to (a) add such support, and (b) maintain such strings.

beldmit commented 3 months ago

[2] is supported in OpenSSL so I strongly prefer this solution. Installing oqsprovider is not a common configuration and anyway implies changing the openssl.cnf file for activation

mouse07410 commented 3 months ago

[2] is supported in OpenSSL so I strongly prefer this solution.

Are you saying that [ssl-signatures] parameter in openssl.cnf is already supported? If so, references, please? Ideally - both to the code that implements it and to the man page that describes it?

Installing oqsprovider is not a common configuration and anyway implies changing the openssl.cnf file for activation

Yes. But even an otherwise-ignorant user like yours-truly can add a few trivial lines to openssl.cnf that enable oqsprovider. And the process is both short enough and well-documented.

While listing all the acceptable signature algorithms, especially when you factor in the providers, is IMHO an insurmountable task - and the one that no one can expect normal users to perform.

So, unless there are "cribs", aka - examples of complete parameter-strings that would preserve the current TLS behavior with the Classic algorithms and give "sub-strings" to append for adding algorithms from providers (probably, for every provider) - I'm strongly against the [2], because it is the best in theory but very much unusable in practice.

beldmit commented 3 months ago

https://openssl.org/docs/man3.2/man5/config.html, SSL Configuration https://openssl.org/docs/man3.2/man3/SSL_CONF_cmd.html

It is here since 1.1.1

But I totally agree that the provider-specific examples of these commands are necessary

mouse07410 commented 3 months ago

https://openssl.org/docs/man3.2/man5/config.html, SSL Configuration

Sorry, I must be dense today - but I did not find anything there that lists signatures algorithms that should be allowed/enabled for TLS. :-(

beldmit commented 3 months ago

Quoting:

Each configuration section consists of name/value pairs that are parsed by SSL_CONF_cmd(3), which will be called by SSL_CTX_config() or SSL_config(), appropriately.

mouse07410 commented 3 months ago

@beldmit thanks, but I still don't understand. I cannot find field sig-signatures, or a similarly-named field anywhere in the OpenSSL codebase (master branch). I noticed the flag -sigalgs apparently available in openssl s_client command-line, but I don't see how it translates to availability in openssl.cnf. Would you mind showing exactly how to add entries to the OpenSSL config file now? And if you know - where to get the complete list of the supported algorithms in a format suitable for inclusion in the config file?

IMHO, yet another example of stuff not being ready for a "normal" user. Because if to accomplish even such a simple step one has to ask experts - it's not ready by my book.

beldmit commented 3 months ago

It's named SignatureAlgorithms. And yes, any documentation contribution to OpenSSL is welcome

mouse07410 commented 3 months ago

It's named SignatureAlgorithms. And yes, any documentation contribution to OpenSSL is welcome

It makes sense that those who understand the subject contribute documentation. E.g., what would you expect me to contrbute - my questions to you?

beldmit commented 3 months ago

Sorry, let me disagree. I'm aware of the structure of the documentation and know where to look for it. If you find not obvious the entry point to the documentation, please let me know what you think to be a valid starting point to look.

mouse07410 commented 3 months ago

If you find not obvious the entry point to the documentation, please let me know what you think to be a valid starting point to look.

Oh sure! Where do I begin? :-)

First, an obvious place for me to learn about the contents of the config file would be (unsurprisingly!) something like man openssl-config or man config. OK, present. I proceed down the page and find [ssl_configuration]. OK, excellent. It further points me at [client_tls_config] and [server_tls_config]. Great - but what are the values that I can put there? Here I stumble. Either there should be a list of all of the supported keys/parameter-names/whatever, or a reference like "to see the list of all the keys do man config-tls". And then - for each key needs to be a list of all the possible/legal values, and/or a way to get it (in the correct/required syntax).

How's that for the beginning? ;-)

beldmit commented 3 months ago

So the config manual page cuurently has the wording

Each configuration section consists of name/value pairs that are parsed by SSL_CONF_cmd(3)

What's wrong here and could you please submit a PR to make it better?

mouse07410 commented 3 months ago

What's wrong here?

The mere fact that neither a list (let alone a complete list) of the keys and their allowed values is given, nor is the format (suitable for inclusion in openssl.cnf) of those mentioned in SSL_CONF_cmd specified. Also, as I said before - lack of the list of all the algorithms in format suitable for direct inclusion in openssl.cnf.

As I also already said, it's impractical to expect a normal user to know those.

could you please submit a PR to make it better?

If I knew the answers to the above questions - I wouldn't bother asking people here and spending my time on this discussion. Instead, I would've just edited openssl.cnf and got done with it. Unfortunately, I don't have those answers - so, here I am. :-(

baentsch commented 3 months ago

What's wrong here and could you please submit a PR to make it better?

First step done in https://github.com/openssl/openssl/pull/24499

@mouse07410 The list of default-active sigalgs is indeed documented in SSL_CONF_cmd(3), though not in a way that it can be written down immediately. Also, it's dependent on the activated providers (as clarified in the PR above just created) which is tedious but a consequence of the dynamic provider concept.

@beldmit Are you aware of a facility within openssl to extract all such (currently registered) algorithm combinations along the lines (but extending to all permitted configurable sigalg combinations) of openssl list -signature-algorithms?

baentsch commented 3 months ago

And another question if I may, @beldmit : Do you know where the semantics of "..." is documented as you used it in your "crypto_policy" example above? When "playing around" with a .cnf file now generated as part of an enhanced oqsprovider generator script, these three dots make a massive difference as to what works and what doesn't.

beldmit commented 3 months ago

@baentsch In Fedora we ship oqsprovider/liboqs and have the corresponding PQ policy. Currently it provides all the algorithms supported by liboqs

baentsch commented 3 months ago

@baentsch In Fedora we ship oqsprovider/liboqs and have the corresponding PQ policy.

Thanks for letting me know, but my question pertains to openssl (config options in general). Should we create a separate discussion item there?

Currently it provides all the algorithms supported by liboqs

So does this mean you disable there all hybrid and composite algs, thus working around this issue by supporting fewer than 64 sigalgs?

mouse07410 commented 3 months ago

The list of default-active sigalgs is indeed documented in SSL_CONF_cmd(3), though not in a way that it can be written down immediately.

Which is one of my points - a user would have to figure out how to write them down in a proper form. I don't think such an expectation is reasonable.

Also, it's dependent on the activated providers (as clarified in the PR above just created) which is tedious but a consequence of the dynamic provider concept.

Which is my other point - we can't get away from dynamic providers (nor do we want to), so the process becomes not merely tedious, but requiring an expert.

I say again - this is a nice theoretic solution that has no room in the ugly practical world.