Odd CLI test failure on MacOS 14 runners

[randombit]

I've seen this twice now:

python3 ./src/scripts/test_cli.py --threads=3 ./botan
     INFO: Ran cli_base32_tests in 0.02 sec
     INFO: Ran cli_asn1_tests in 0.03 sec
     INFO: Ran cli_argon2_tests in 0.05 sec
     INFO: Ran cli_base64_tests in 0.01 sec
     INFO: Ran cli_base58_tests in 0.03 sec
     INFO: Ran cli_cycle_counter in 0.01 sec
     INFO: Ran cli_bcrypt_tests in 0.02 sec
     INFO: Ran cli_cc_enc_tests in 0.05 sec
     INFO: Ran cli_compress_tests in 0.02 sec
     INFO: Ran cli_config_tests in 0.02 sec
     INFO: Ran cli_cpuid_tests in 0.06 sec
     INFO: Ran cli_dl_group_info_tests in 0.07 sec
     INFO: Ran cli_cert_issuance_tests in 0.22 sec
     INFO: Ran cli_entropy_tests in 0.01 sec
     INFO: Ran cli_ec_group_info_tests in 0.02 sec
     INFO: Ran cli_cert_issuance_alternative_algos_tests in 0.27 sec
     INFO: Ran cli_gen_prime_tests in 0.01 sec
     INFO: Ran cli_hash_tests in 0.03 sec
     INFO: Ran cli_help_tests in 0.01 sec
     INFO: Ran cli_hex_tests in 0.01 sec
     INFO: Ran cli_hmac_tests in 0.01 sec
     INFO: Ran cli_is_prime_tests in 0.02 sec
     INFO: Ran cli_factor_tests in 0.15 sec
     INFO: Ran cli_marvin_tests in 0.06 sec
     INFO: Ran cli_mod_inverse_tests in 0.01 sec
     INFO: Ran cli_gen_dl_group_tests in 0.33 sec
     INFO: Ran cli_pk_encrypt_tests in 0.10 sec
     INFO: Ran cli_pk_workfactor_tests in 0.03 sec
     INFO: Ran cli_psk_db_tests in 0.06 sec
     INFO: Ran cli_rng_tests in 0.06 sec
     INFO: Ran cli_roughtime_check_tests in 0.03 sec
     INFO: Ran cli_roughtime_tests in 0.05 sec
     INFO: Ran cli_pbkdf_tune_tests in 0.55 sec
     INFO: Ran cli_tls_ciphersuite_tests in 0.04 sec
     INFO: Ran cli_tls_client_hello_tests in 0.03 sec
     INFO: Ran cli_timing_test_tests in 0.39 sec
     INFO: Ran cli_key_tests in 1.28 sec
     INFO: Ran cli_tls_http_server_tests in 1.22 sec
     INFO: Ran cli_tls_online_pqc_hybrid_tests in 0.00 sec
     INFO: Ran cli_trust_root_tests in 0.52 sec
     INFO: Ran cli_tss_tests in 0.06 sec
     INFO: Ran cli_uuid_tests in 0.01 sec
     INFO: Ran cli_version_tests in 0.01 sec
     INFO: Ran cli_zfec_tests in 0.06 sec
     INFO: Ran cli_tls_proxy_tests in 6.27 sec
     INFO: Ran cli_tls_socket_tests in 22.73 sec
  Ran 232 tests with 0 failures in 24.46 seconds
  Ran for 24 seconds
  Command 'python3 ./src/scripts/test_cli.py --threads=3 ./botan' failed with error code -11

https://github.com/randombit/botan/actions/runs/8614981865/job/23609543455?pr=3990

The "Ran 232 tests" line is the last thing the script prints before it returns, so it looks like Python is crashing with a segfault on exit. Maybe a bug in multiprocessing affecting Mac on Aarch64?

[randombit]

Possibly related: https://bugs.python.org/issue33725

[randombit]

Oh smoking gun in the multiprocessing docs:

Changed in version 3.8: On macOS, the spawn start method is now the default. The fork start method should be considered unsafe as it can lead to crashes of the subprocess.

[randombit]

Oh but we're running on Python 3.12 in CI ...

[randombit]

Recurred again in https://github.com/randombit/botan/actions/runs/9965253706/job/27535202682 so #4216 was not the fix

[reneme]

Tried running the CLI tests repeatedly with multiple threads on my M2 mac w/ macOS 14.5 and Python 3.12.4. No luck. :(