Closed junaruga closed 11 months ago
Though I am not sure if it is related to this issue, I found a kind of bug for the openssl
command generating an invalid pem key with the DH 2048 in FIPS, and reported the issue https://github.com/openssl/openssl/issues/22620 in OpenSSL project.
I think this is a spec. man EVP_PKEY-DH
says:
DH additional domain parameters
"safeprime-generator" (OSSL_PKEY_PARAM_DH_GENERATOR) <integer>
Used for DH generation of safe primes using the old safe prime generator code. The default value is 2. It is
recommended to use a named safe prime group instead, if domain parameter validation is required.
Randomly generated safe primes are not allowed by FIPS, so setting this value for the OpenSSL FIPS provider will
instead choose a named safe prime group based on the size of p.
Randomly generated safe primes are not allowed by FIPS, so setting this value for the OpenSSL FIPS provider will instead choose a named safe prime group based on the size of p.
Thank you for telling me the info. I was still not able to understand it.
Checking the difference of the behavior between non-FIPS and FIPS.
In the code above, the line 357,
In the FIPS case, the following EVP_PKEY_paramgen
returns 1
, and the pkey_blocking_gen
returns arg->pkey
.
(gdb) p EVP_PKEY_paramgen(arg->ctx, &arg->pkey)
$12 = 1
In the non-FIPS case, the following function EVP_PKEY_paramgen
returns 0
, and pkey_blocking_gen
returns NULL
.
(gdb) p EVP_PKEY_paramgen(arg->ctx, &arg->pkey)
$2 = 0
I still don't understand why the document you mentioned is related to the difference of the return value of the EVP_PKEY_paramgen
. Could you tell me more about it?
By the way, the generated key in the following command, is always same content in my environment. The issue is managed on https://github.com/openssl/openssl/issues/22620 . I am not sure if this is a spec or a bug.
$ OPENSSL_CONF=/home/jaruga/.local/openssl-3.3.0-dev-fips-debug-1aa08644ec/ssl/openssl_fips.cnf \
ruby -I./lib -ropenssl <<EOF
pkey = OpenSSL::PKey.generate_parameters("DH", {"dh_paramgen_prime_len" => 2048, "dh_paramgen_generator" => 2}) { break }
puts pkey.to_s
EOF
-----BEGIN DH PARAMETERS-----
MIIBCAKCAQEA//////////+t+FRYortKmq/cViAnPTzx2LnFg84tNpWp4TZBFGQz
+8yTnc4kmz75fS/jY2MMddj2gbICrsRhetPfHtXV/WVhJDP1H18GbtCFY2VVPe0a
87VXE15/V8k1mE8McODmi3fipona8+/och3xWKE2rec1MKzKT0g6eXq8CrGCsyT7
YdEIqUuyyOP7uWrat2DX9GgdT0Kj3jlN9K5W7edjcrsZCwenyO4KbXCeAvzhzffi
7MA0BM0oNC9hkXL+nOmFg/+OTxIy7vKBg8P+OxtMb61zO7X8vC7CIAXFjvGDfRaD
ssbzSibBsu/6iGtCOGEoXJf//////////wIBAg==
-----END DH PARAMETERS-----
I still don't understand why the document you mentioned is related to the difference of the return value of the EVP_PKEY_paramgen.
The block passed to OpenSSL::PKey.generate_parameters
is set to EVP_PKEY_CTX_set_cb
, which in turn is invoked from the lower-level API DH_generate_parameters_ex()
whenever a potential prime number is found (I think).
Normally, this callback pkey_gen_cb()
is called at least once because generating DH group parameters involves generating prime numbers. In the test case, the passed block does break
. This is caught by rb_protect()
and causes the callback to return 0. The parameter generation code in OpenSSL checks the return value of the callback and quits early. EVP_PKEY_paramgen()
returns 0 indicating a failure.
But OpenSSL apparently doesn't generate new prime numbers in the FIPS mode, so the callback is never called, and EVP_PKEY_paramgen() succeeds. This is shown below:
By the way, the generated key in the following command, is always same content in my environment. The issue is managed on openssl/openssl#22620 . I am not sure if this is a spec or a bug.
$ OPENSSL_CONF=/home/jaruga/.local/openssl-3.3.0-dev-fips-debug-1aa08644ec/ssl/openssl_fips.cnf \ ruby -I./lib -ropenssl <<EOF pkey = OpenSSL::PKey.generate_parameters("DH", {"dh_paramgen_prime_len" => 2048, "dh_paramgen_generator" => 2}) { break } puts pkey.to_s EOF -----BEGIN DH PARAMETERS----- MIIBCAKCAQEA//////////+t+FRYortKmq/cViAnPTzx2LnFg84tNpWp4TZBFGQz +8yTnc4kmz75fS/jY2MMddj2gbICrsRhetPfHtXV/WVhJDP1H18GbtCFY2VVPe0a 87VXE15/V8k1mE8McODmi3fipona8+/och3xWKE2rec1MKzKT0g6eXq8CrGCsyT7 YdEIqUuyyOP7uWrat2DX9GgdT0Kj3jlN9K5W7edjcrsZCwenyO4KbXCeAvzhzffi 7MA0BM0oNC9hkXL+nOmFg/+OTxIy7vKBg8P+OxtMb61zO7X8vC7CIAXFjvGDfRaD ssbzSibBsu/6iGtCOGEoXJf//////////wIBAg== -----END DH PARAMETERS-----
This is ffdhe2048 from RFC 7919, so not generated by OpenSSL. (We happen to have a copy of this exact PEM encoding in https://github.com/ruby/openssl/pull/674).
I haven't checked the relevant documentations for FIPS 140 and don't know why OpenSSL doesn't/can't generate new group parameters in the FIPS mode, but it seems intentional.
I still don't understand why the document you mentioned is related to the difference of the return value of the EVP_PKEY_paramgen.
The block passed to
OpenSSL::PKey.generate_parameters
is set toEVP_PKEY_CTX_set_cb
, which in turn is invoked from the lower-level APIDH_generate_parameters_ex()
whenever a potential prime number is found (I think).Normally, this callback
pkey_gen_cb()
is called at least once because generating DH group parameters involves generating prime numbers. In the test case, the passed block doesbreak
. This is caught byrb_protect()
and causes the callback to return 0. The parameter generation code in OpenSSL checks the return value of the callback and quits early.EVP_PKEY_paramgen()
returns 0 indicating a failure.But OpenSSL apparently doesn't generate new prime numbers in the FIPS mode, so the callback is never called, and EVP_PKEY_paramgen() succeeds. This is shown below:
Okay. That meant "Randomly generated safe primes are not allowed by FIPS" in the man 7 EVP_PKEY-DH
EVP_PKEY-DH(7)
.
By the way, the generated key in the following command, is always same content in my environment. The issue is managed on openssl/openssl#22620 . I am not sure if this is a spec or a bug.
$ OPENSSL_CONF=/home/jaruga/.local/openssl-3.3.0-dev-fips-debug-1aa08644ec/ssl/openssl_fips.cnf \ ruby -I./lib -ropenssl <<EOF pkey = OpenSSL::PKey.generate_parameters("DH", {"dh_paramgen_prime_len" => 2048, "dh_paramgen_generator" => 2}) { break } puts pkey.to_s EOF -----BEGIN DH PARAMETERS----- MIIBCAKCAQEA//////////+t+FRYortKmq/cViAnPTzx2LnFg84tNpWp4TZBFGQz +8yTnc4kmz75fS/jY2MMddj2gbICrsRhetPfHtXV/WVhJDP1H18GbtCFY2VVPe0a 87VXE15/V8k1mE8McODmi3fipona8+/och3xWKE2rec1MKzKT0g6eXq8CrGCsyT7 YdEIqUuyyOP7uWrat2DX9GgdT0Kj3jlN9K5W7edjcrsZCwenyO4KbXCeAvzhzffi 7MA0BM0oNC9hkXL+nOmFg/+OTxIy7vKBg8P+OxtMb61zO7X8vC7CIAXFjvGDfRaD ssbzSibBsu/6iGtCOGEoXJf//////////wIBAg== -----END DH PARAMETERS-----
This is ffdhe2048 from RFC 7919, so not generated by OpenSSL. (We happen to have a copy of this exact PEM encoding in #674).
I haven't checked the relevant documentations for FIPS 140 and don't know why OpenSSL doesn't/can't generate new group parameters in the FIPS mode, but it seems intentional.
Oh, thank you for explaining it! Now I was able to understand in a better way.
I can see the following hexadecimal representation of p is used to output the
RFC 7919 - ffdhe2048 - "The hexadecimal representation of p" https://www.rfc-editor.org/rfc/rfc7919#appendix-A.1
$ ruby -I./lib -ropenssl <<EOF
print OpenSSL::PKey.read(OpenSSL::ASN1::Sequence([OpenSSL::ASN1::Integer((<<~END).split.join.to_i(16)), OpenSSL::ASN1::Integer(2)]).to_der).to_pem
FFFFFFFF FFFFFFFF ADF85458 A2BB4A9A AFDC5620 273D3CF1
D8B9C583 CE2D3695 A9E13641 146433FB CC939DCE 249B3EF9
7D2FE363 630C75D8 F681B202 AEC4617A D3DF1ED5 D5FD6561
2433F51F 5F066ED0 85636555 3DED1AF3 B557135E 7F57C935
984F0C70 E0E68B77 E2A689DA F3EFE872 1DF158A1 36ADE735
30ACCA4F 483A797A BC0AB182 B324FB61 D108A94B B2C8E3FB
B96ADAB7 60D7F468 1D4F42A3 DE394DF4 AE56EDE7 6372BB19
0B07A7C8 EE0A6D70 9E02FCE1 CDF7E2EC C03404CD 28342F61
9172FE9C E98583FF 8E4F1232 EEF28183 C3FE3B1B 4C6FAD73
3BB5FCBC 2EC22005 C58EF183 7D1683B2 C6F34A26 C1B2EFFA
886B4238 61285C97 FFFFFFFF FFFFFFFF
END
EOF
-----BEGIN DH PARAMETERS-----
MIIBCAKCAQEA//////////+t+FRYortKmq/cViAnPTzx2LnFg84tNpWp4TZBFGQz
+8yTnc4kmz75fS/jY2MMddj2gbICrsRhetPfHtXV/WVhJDP1H18GbtCFY2VVPe0a
87VXE15/V8k1mE8McODmi3fipona8+/och3xWKE2rec1MKzKT0g6eXq8CrGCsyT7
YdEIqUuyyOP7uWrat2DX9GgdT0Kj3jlN9K5W7edjcrsZCwenyO4KbXCeAvzhzffi
7MA0BM0oNC9hkXL+nOmFg/+OTxIy7vKBg8P+OxtMb61zO7X8vC7CIAXFjvGDfRaD
ssbzSibBsu/6iGtCOGEoXJf//////////wIBAg==
-----END DH PARAMETERS-----
OK. Now I can close this ticket as it looks intentional. Thank you for your explanation!
The block passed to OpenSSL::PKey.generate_parameters is set to EVP_PKEY_CTX_set_cb, which in turn is invoked from the lower-level API DH_generate_parameters_ex() whenever a potential prime number is found (I think).
I debugged this now in OpenSSL code (openssl/openssl@1aa08644ecd4005c0f55276b2e8dabd8a2a758f0)
In the following code block, the first if
block (line 710) is executed in FIPS case. But the else
block (line 721) where the DH_generate_parameters_ex
exists, is executed in non-FIPS case.
$ pwd
/home/jaruga/git/ruby/openssl
$ cat test.rb
o = OpenSSL::PKey.generate_parameters("DH", {"dh_paramgen_prime_len" => 2048, "dh_paramgen_generator" => 2}) { break }
p o
$ OPENSSL_CONF=/home/jaruga/.local/openssl-3.3.0-dev-fips-debug-1aa08644ec/ssl/openssl_fips.cnf \
gdb --args ruby -I./lib -ropenssl test.rb
<snip>
(gdb) f
#0 dh_gen (genctx=0x7cf000, osslcb=0x7fffce44f9ba <ossl_callback_to_pkey_gencb>,
cbarg=0x808a50) at providers/implementations/keymgmt/dh_kmgmt.c:720
720 ffc = ossl_dh_get0_params(dh);
(gdb) bt
#0 dh_gen (genctx=0x7cf000, osslcb=0x7fffce44f9ba <ossl_callback_to_pkey_gencb>, cbarg=0x808a50) at providers/implementations/keymgmt/dh_kmgmt.c:720
#1 0x00007fffce442862 in evp_keymgmt_gen (keymgmt=0x649020, genctx=0x7cf000, cb=0x7fffce44f9ba <ossl_callback_to_pkey_gencb>, cbarg=0x808a50) at crypto/evp/keymgmt_meth.c:384
#2 0x00007fffce441666 in evp_keymgmt_util_gen (target=0x7ae2b0, keymgmt=0x649020, genctx=0x7cf000, cb=0x7fffce44f9ba <ossl_callback_to_pkey_gencb>, cbarg=0x808a50) at crypto/evp/keymgmt_lib.c:513
#3 0x00007fffce44fc44 in EVP_PKEY_generate (ctx=0x808a50, ppkey=0x7fffffffc6d8) at crypto/evp/pmeth_gn.c:189
#4 0x00007fffce44fe93 in EVP_PKEY_paramgen (ctx=0x808a50, ppkey=0x7fffffffc6d8) at crypto/evp/pmeth_gn.c:265
#5 0x00007fffceb4d88f in pkey_blocking_gen (ptr=0x7fffffffc6d0) at ../../../../ext/openssl/ossl_pkey.c:357
#6 0x00007fffceb4db50 in pkey_generate (argc=2, argv=0x7ffff7440050, self=140736661834360, genparam=1) at ../../../../ext/openssl/ossl_pkey.c:431
#7 0x00007fffceb4dc09 in ossl_pkey_s_generate_parameters (argc=2, argv=0x7ffff7440050, self=140736661834360) at ../../../../ext/openssl/ossl_pkey.c:472
#8 0x00007ffff7af5ce0 in ractor_safe_call_cfunc_m1 (recv=140736661834360, argc=2, argv=0x7ffff7440050, func=0x7fffceb4dbdf <ossl_pkey_s_generate_parameters>) at vm_insnhelper.c:3312
#9 0x00007ffff7af6931 in vm_call_cfunc_with_frame_ (ec=0x40a320, reg_cfp=0x7ffff753ffa0, calling=0x7fffffffcb90, argc=2, argv=0x7ffff7440050, stack_bottom=0x7ffff7440048) at vm_insnhelper.c:3503
#10 0x00007ffff7af6b55 in vm_call_cfunc_with_frame (ec=0x40a320, reg_cfp=0x7ffff753ffa0, calling=0x7fffffffcb90) at vm_insnhelper.c:3531
#11 0x00007ffff7af6c69 in vm_call_cfunc_other (ec=0x40a320, reg_cfp=0x7ffff753ffa0, calling=0x7fffffffcb90) at vm_insnhelper.c:3557
#12 0x00007ffff7af7084 in vm_call_cfunc (ec=0x40a320, reg_cfp=0x7ffff753ffa0, calling=0x7fffffffcb90) at vm_insnhelper.c:3639
#13 0x00007ffff7af97e1 in vm_call_method_each_type (ec=0x40a320, cfp=0x7ffff753ffa0, calling=0x7fffffffcb90) at vm_insnhelper.c:4417
#14 0x00007ffff7afa285 in vm_call_method (ec=0x40a320, cfp=0x7ffff753ffa0, calling=0x7fffffffcb90) at vm_insnhelper.c:4543
#15 0x00007ffff7afa483 in vm_call_general (ec=0x40a320, reg_cfp=0x7ffff753ffa0, calling=0x7fffffffcb90) at vm_insnhelper.c:4587
#16 0x00007ffff7afcc33 in vm_sendish (ec=0x40a320, reg_cfp=0x7ffff753ffa0, cd=0x4c1730, block_handler=140737342865337, method_explorer=mexp_search_method) at vm_insnhelper.c:5581
#17 0x00007ffff7b03f68 in vm_exec_core (ec=0x40a320) at insns.def:802
#18 0x00007ffff7b19f3b in rb_vm_exec (ec=0x40a320) at vm.c:2472
#19 0x00007ffff7b1add3 in rb_iseq_eval_main (iseq=0x7fffcebb3070) at vm.c:2738
#20 0x00007ffff790686b in rb_ec_exec_node (ec=0x40a320, n=0x7fffcebb3070) at eval.c:287
#21 0x00007ffff79069cc in ruby_run_node (n=0x7fffcebb3070) at eval.c:328
#22 0x00000000004011d9 in rb_main (argc=4, argv=0x7fffffffdb18) at ./main.c:39
#23 0x0000000000401236 in main (argc=4, argv=0x7fffffffdb18) at ./main.c:58
<snip>
(gdb) n
780 ret = 1;
(gdb) p dh
$6 = (DH *) 0x7b08a0
<snip>
(gdb) n
786 BN_GENCB_free(gencb);
(gdb) p ret
$7 = 1
The DH_generate_parameters_ex
is executed, and fails with ret
as 0
, then dh
is freed, and set as NULL
in the line 783, 784.
$ gdb --args ruby -I./lib -ropenssl test.rb
<snip>
(gdb) f
#0 dh_gen (genctx=0x72db60, osslcb=0x7fffce44f9ba <ossl_callback_to_pkey_gencb>,
cbarg=0x5e51f0) at providers/implementations/keymgmt/dh_kmgmt.c:756
756 ret = DH_generate_parameters_ex(dh, gctx->pbits,
(gdb) n
762 if (ret <= 0)
(gdb) p ret
$7 = 0
(gdb) bt
#0 dh_gen (genctx=0x72db60, osslcb=0x7fffce44f9ba <ossl_callback_to_pkey_gencb>, cbarg=0x5e51f0) at providers/implementations/keymgmt/dh_kmgmt.c:762
#1 0x00007fffce442862 in evp_keymgmt_gen (keymgmt=0x6b0180, genctx=0x72db60, cb=0x7fffce44f9ba <ossl_callback_to_pkey_gencb>, cbarg=0x5e51f0) at crypto/evp/keymgmt_meth.c:384
#2 0x00007fffce441666 in evp_keymgmt_util_gen (target=0x553050, keymgmt=0x6b0180, genctx=0x72db60, cb=0x7fffce44f9ba <ossl_callback_to_pkey_gencb>, cbarg=0x5e51f0) at crypto/evp/keymgmt_lib.c:513
#3 0x00007fffce44fc44 in EVP_PKEY_generate (ctx=0x5e51f0, ppkey=0x7fffffffc758) at crypto/evp/pmeth_gn.c:189
#4 0x00007fffce44fe93 in EVP_PKEY_paramgen (ctx=0x5e51f0, ppkey=0x7fffffffc758) at crypto/evp/pmeth_gn.c:265
#5 0x00007fffceb4d88f in pkey_blocking_gen (ptr=0x7fffffffc750) at ../../../../ext/openssl/ossl_pkey.c:357
#6 0x00007fffceb4db50 in pkey_generate (argc=2, argv=0x7ffff7440050, self=140736661834360, genparam=1) at ../../../../ext/openssl/ossl_pkey.c:431
#7 0x00007fffceb4dc09 in ossl_pkey_s_generate_parameters (argc=2, argv=0x7ffff7440050, self=140736661834360) at ../../../../ext/openssl/ossl_pkey.c:472
#8 0x00007ffff7af5ce0 in ractor_safe_call_cfunc_m1 (recv=140736661834360, argc=2, argv=0x7ffff7440050, func=0x7fffceb4dbdf <ossl_pkey_s_generate_parameters>) at vm_insnhelper.c:3312
#9 0x00007ffff7af6931 in vm_call_cfunc_with_frame_ (ec=0x40a320, reg_cfp=0x7ffff753ffa0, calling=0x7fffffffcc10, argc=2, argv=0x7ffff7440050, stack_bottom=0x7ffff7440048) at vm_insnhelper.c:3503
#10 0x00007ffff7af6b55 in vm_call_cfunc_with_frame (ec=0x40a320, reg_cfp=0x7ffff753ffa0, calling=0x7fffffffcc10) at vm_insnhelper.c:3531
#11 0x00007ffff7af6c69 in vm_call_cfunc_other (ec=0x40a320, reg_cfp=0x7ffff753ffa0, calling=0x7fffffffcc10) at vm_insnhelper.c:3557
#12 0x00007ffff7af7084 in vm_call_cfunc (ec=0x40a320, reg_cfp=0x7ffff753ffa0, calling=0x7fffffffcc10) at vm_insnhelper.c:3639
#13 0x00007ffff7af97e1 in vm_call_method_each_type (ec=0x40a320, cfp=0x7ffff753ffa0, calling=0x7fffffffcc10) at vm_insnhelper.c:4417
#14 0x00007ffff7afa285 in vm_call_method (ec=0x40a320, cfp=0x7ffff753ffa0, calling=0x7fffffffcc10) at vm_insnhelper.c:4543
#15 0x00007ffff7afa483 in vm_call_general (ec=0x40a320, reg_cfp=0x7ffff753ffa0, calling=0x7fffffffcc10) at vm_insnhelper.c:4587
#16 0x00007ffff7afcc33 in vm_sendish (ec=0x40a320, reg_cfp=0x7ffff753ffa0, cd=0x72d250, block_handler=140737342865337, method_explorer=mexp_search_method) at vm_insnhelper.c:5581
#17 0x00007ffff7b03f68 in vm_exec_core (ec=0x40a320) at insns.def:802
#18 0x00007ffff7b19f3b in rb_vm_exec (ec=0x40a320) at vm.c:2472
#19 0x00007ffff7b1add3 in rb_iseq_eval_main (iseq=0x7fffcebb3070) at vm.c:2738
#20 0x00007ffff790686b in rb_ec_exec_node (ec=0x40a320, n=0x7fffcebb3070) at eval.c:287
#21 0x00007ffff79069cc in ruby_run_node (n=0x7fffcebb3070) at eval.c:328
#22 0x00000000004011d9 in rb_main (argc=4, argv=0x7fffffffdb98) at ./main.c:39
#23 0x0000000000401236 in main (argc=4, argv=0x7fffffffdb98) at ./main.c:58
<snip>
(gdb) n
783 DH_free(dh);
(gdb) n
784 dh = NULL;
(gdb) n
786 BN_GENCB_free(gencb);
(gdb) p dh
$8 = (DH *) 0x0
For the following assertion in FIPS case, I am seeing an interesting difference of the behavior between non-FIPS and FIPS.
https://github.com/ruby/openssl/blob/c9b48f989b25a45bae1e6feb2673da93090670df/test/openssl/test_pkey_dh.rb#L22
My environment
My environment is below. Fedora 38, with relatively latest ruby master branch and OpenSSL master branch.
I compiled like this.
The issue summary
Then here is the difference. The
OpenSSL::PKey::DH.new(2048) { break }
is expected to returnnil
. However, in FIPS, it returns a value. TheOpenSSL::PKey::DH.new
callsOpenSSL::PKey.generate_parameters
in it. I am still debugging. But I appreciate if you can let me know this is an expected behavior or a kind of bug.In Non-FIPS
In FIPS