X509 PKXI::check_ocsp_online: check for server timeout without effect

[fweissberg]

When using OCSP for getting revokation information on X509 certificates, Botan may wait idefinitely on a response from the server.

PKXI::check_ocsp_online (in x509path.cpp) checks if the std::futures created locally for contacting the servers time out, but does not stop them. (the call to wait_for(timeout) does not terminate the futures). When the function tries to return, the local std::future objects need to be deconstructed. Apparently, this does not happen while the associated threads are still running/waiting.

How to reproduce with a local http server and an existing test case:

• Setup a minimal http server that enters an inifite loop when serving http posts (e.g. the minimal python server from the attachment)
• Modify PKXI::check_ocsp_online to always query the localhost for ocsp requests (e.g. with the git patch from the attachment that hardcodes http://127.0.0.1 )
• Build and execute ./botan-test ocsp --run-online-tests
• Notice that the test finishes almost instantly when the server is down in the first place or it is killed mid-test

Tested with and without boost support on Ubuntu 16.04 (output from configure below (example with boost))

./configure.py --with-boost INFO: ./configure.py invoked with options "--with-boost" INFO: Platform: OS="Linux" machine="x86_64" proc="x86_64" INFO: Guessing target OS is linux (use --os to set) INFO: Guessing to use compiler gcc (use --cc to set) INFO: Detected CPU model "Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz" in /proc/cpuinfo INFO: Guessing target processor is a x86_64/x86_64 (use --cpu to set) INFO: Canonicalized CPU target x86_64 to x86_64/x86_64 INFO: Auto-detected compiler version 5.4 INFO: Target is gcc:5.4-linux-x86_64-x86_64 INFO: Skipping (dependency failure): certstor_sqlite3 sessions_sqlite3 INFO: Skipping (incompatible CPU): aes_armv8 pmull sha1_armv8 sha2_32_armv8 INFO: Skipping (incompatible OS): darwin_secrandom getentropy win32_stats INFO: Skipping (no enabled compression schemes): compression INFO: Skipping (requires external dependency): bearssl bzip2 lzma openssl sqlite3 tpm zlib INFO: Enabling use of external dependency boost INFO: Loa

ding modules: adler32 aead aes aes_ni aes_ssse3 aont aria asn1 auto_rng base base64 bcrypt bigint blake2 block blowfish boost camellia cascade cast cbc cbc_mac ccm cecpq1 certstor_sql cfb chacha chacha20poly1305 chacha_rng chacha_sse2 clmul clmul_ssse3 cmac codec_filt comb4p cpuid crc24 crc32 cryptobox ctr curve25519 des dev_random dh dl_algo dl_group dlies dsa dyn_load eax ec_gfp ec_group ecc_key ecdh ecdsa ecgdsa ecies eckcdsa ed25519 elgamal eme_oaep eme_pkcs1 eme_raw emsa1 emsa_pkcs1 emsa_pssr emsa_raw emsa_x931 entropy fd_unix ffi filters fpe_fe1 gcm gmac gost_28147 gost_3410 gost_3411 hash hash_id hex hkdf hmac hmac_drbg hotp http_util idea idea_sse2 iso9796 kasumi kdf kdf1 kdf1_iso18033 kdf2 keccak keypair lion locking_allocator mac mce mceies md4 md5 mdx_hash mgf1 misty1 mode_pad modes mp newhope nist_keywrap noekeon noekeon_simd numbertheory ocb ofb par_hash passhash9 pbes2 pbkdf pbkdf1 pbkdf2 pem pgp_s2k pk_pad pkcs11 poly1305 poly_dbl prf_tls prf_x942 proc_walk psk_db pubkey rc4 rdrand rdrand_rng rdseed rfc3394 rfc6979 rmd160 rng rsa salsa20 seed serpent serpent_simd sessions_sql sha1 sha1_sse2 sha1_x86 sha2_32 sha2_32_x86 sha2_64 sha3 shacal2 shacal2_simd shacal2_x86 shake shake_cipher simd siphash siv skein sm2 sm3 sm4 sp800_108 sp800_56a sp800_56c srp6 stateful_rng stream streebog system_rng threefish threefish_avx2 tiger tls tls_cbc tss twofish utils whirlpool x509 x919_mac xmss xtea xts INFO: Assuming CPU is little endian INFO: Assuming unaligned memory access works INFO: Using symlink to link files into build dir (use --link-method to change) INFO: Botan 2.4.0 (revision git:a201a07ea17b003bd0b373eadbfbc7711610505e) (unreleased undated) build setup is complete apathic_server.tar.gz

[randombit]

Ouch. I think we'll need to restructure how HTTP requests are issued, in order to handle this case.

[randombit]

I thought about this a little more, I think the thing to do is to pass the desired timeout down to the HTTP request layer itself. That will require some refactoring but can be done without breaking any public APIs. The socket API is internal so we can do what we want. Biggest trouble is probably going to be implementing the timeout logic for all 3 of asio, BSD sockets, and WinSock.

[randombit]

@fweissberg With #1360 merged there are timeouts to all HTTP operations and we don't rely on trying to poll or wait_for the futures during path validation.