Add CMake build system (with Windows support)

divergentdave commented 2 years ago

This adds a cross-platform CMake build system, which supports basic building and testing on Linux, macOS and Windows. (It does not yet have parity on Linux/macOS debug builds, sanitizer builds, or running test harnesses, but those could be added in the future with CMAKE_BUILD_TYPE conditionals and CTest. It also doesn't cover the install step yet.) I also rewrote the bash test harnesses in Python, as a cross-platform substitute, and fixed a couple portability issues with the test programs.

Added CMake projects.
Added Python versions of shell scripts.
Moved Windows compatibility defines to common.h, so common.c would see them. (Prior to this, read_cb was checking errno rather than WSAGetLastError(), failing to properly handle WSAEWOULDBLOCK, and reporting an EOF too early.
Added a forward declaration of ws_strerror(). Without this declaration, the compiler would assume it returned an int rather than a pointer, which resulted in the pointer getting truncated, and a later fprintf call performing a wild read. (In retrospect, this was foretold in #235)
Added CMake CI jobs on Windows, macOS, and Linux.

Fixes #192, fixes #121

jsha commented 2 years ago

Excellent, thanks! I'll review this soon.

divergentdave commented 2 years ago

I'll rebase and force push to pick up the Valgrind CI fix

jsha commented 2 years ago

Locally, I ran cmake -S . -B build, then cmake --build build --config Debug --verbose and got:

/home/jsha/.local/lib/python3.9/site-packages/cmake/data/bin/cmake -S/home/jsha/rustls-ffi -B/home/jsha/rustls-ffi/build --check-build-system CMakeFiles/Makefile.cmake 0
/home/jsha/.local/lib/python3.9/site-packages/cmake/data/bin/cmake -E cmake_progress_start /home/jsha/rustls-ffi/build/CMakeFiles /home/jsha/rustls-ffi/build//CMakeFiles/progress.marks
/usr/bin/gmake  -f CMakeFiles/Makefile2 all
gmake[1]: Entering directory '/home/jsha/rustls-ffi/build'
/usr/bin/gmake  -f CMakeFiles/rustls-ffi.dir/build.make CMakeFiles/rustls-ffi.dir/depend
gmake[2]: Entering directory '/home/jsha/rustls-ffi/build'
cd /home/jsha/rustls-ffi/build && /home/jsha/.local/lib/python3.9/site-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /home/jsha/rustls-ffi /home/jsha/rustls-ffi /home/jsha/rustls-ffi/build /home/jsha/rustls-ffi/build /home/jsha/rustls-ffi/build/CMakeFiles/rustls-ffi.dir/DependInfo.cmake --color=
gmake[2]: Leaving directory '/home/jsha/rustls-ffi/build'
/usr/bin/gmake  -f CMakeFiles/rustls-ffi.dir/build.make CMakeFiles/rustls-ffi.dir/build
gmake[2]: Entering directory '/home/jsha/rustls-ffi/build'
[  6%] Performing build step for 'rustls-ffi'
cd /home/jsha/rustls-ffi/build/rust/src/rustls-ffi-build && cbindgen --lang C --output /home/jsha/rustls-ffi/src/rustls.h /home/jsha/rustls-ffi
cd /home/jsha/rustls-ffi/build/rust/src/rustls-ffi-build && cargo build --
    Finished dev [unoptimized + debuginfo] target(s) in 0.03s
[ 13%] No install step for 'rustls-ffi'
cd /home/jsha/rustls-ffi/build/rust/src/rustls-ffi-build && /home/jsha/.local/lib/python3.9/site-packages/cmake/data/bin/cmake -E echo_append
cd /home/jsha/rustls-ffi/build/rust/src/rustls-ffi-build && /home/jsha/.local/lib/python3.9/site-packages/cmake/data/bin/cmake -E touch /home/jsha/rustls-ffi/build/rust/src/rustls-ffi-stamp/rustls-ffi-install
[ 20%] Performing test step for 'rustls-ffi'
cd /home/jsha/rustls-ffi/build/rust/src/rustls-ffi-build && cargo test -- --quiet
    Finished test [unoptimized + debuginfo] target(s) in 0.04s
     Running unittests (/home/jsha/rustls-ffi/target/debug/deps/rustls_ffi-d4abca7efa0e7c2e)

running 27 tests
...........................
test result: ok. 27 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests rustls_ffi

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

cd /home/jsha/rustls-ffi/build/rust/src/rustls-ffi-build && /home/jsha/.local/lib/python3.9/site-packages/cmake/data/bin/cmake -E touch /home/jsha/rustls-ffi/build/rust/src/rustls-ffi-stamp/rustls-ffi-test
[ 26%] Completed 'rustls-ffi'
/home/jsha/.local/lib/python3.9/site-packages/cmake/data/bin/cmake -E make_directory /home/jsha/rustls-ffi/build/CMakeFiles
/home/jsha/.local/lib/python3.9/site-packages/cmake/data/bin/cmake -E touch /home/jsha/rustls-ffi/build/CMakeFiles/rustls-ffi-complete
/home/jsha/.local/lib/python3.9/site-packages/cmake/data/bin/cmake -E touch /home/jsha/rustls-ffi/build/rust/src/rustls-ffi-stamp/rustls-ffi-done
gmake[2]: Leaving directory '/home/jsha/rustls-ffi/build'
[ 60%] Built target rustls-ffi
/usr/bin/gmake  -f tests/CMakeFiles/client.dir/build.make tests/CMakeFiles/client.dir/depend
gmake[2]: Entering directory '/home/jsha/rustls-ffi/build'
cd /home/jsha/rustls-ffi/build && /home/jsha/.local/lib/python3.9/site-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /home/jsha/rustls-ffi /home/jsha/rustls-ffi/tests /home/jsha/rustls-ffi/build /home/jsha/rustls-ffi/build/tests /home/jsha/rustls-ffi/build/tests/CMakeFiles/client.dir/DependInfo.cmake --color=
gmake[2]: Leaving directory '/home/jsha/rustls-ffi/build'
/usr/bin/gmake  -f tests/CMakeFiles/client.dir/build.make tests/CMakeFiles/client.dir/build
gmake[2]: Entering directory '/home/jsha/rustls-ffi/build'
[ 66%] Linking C executable client
cd /home/jsha/rustls-ffi/build/tests && /home/jsha/.local/lib/python3.9/site-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/client.dir/link.txt --verbose=1
/usr/bin/cc CMakeFiles/client.dir/client.c.o CMakeFiles/client.dir/common.c.o -o client
/usr/bin/ld: CMakeFiles/client.dir/client.c.o: in function `do_read':
client.c:(.text+0x202): undefined reference to `rustls_connection_read_tls'
/usr/bin/ld: client.c:(.text+0x287): undefined reference to `rustls_connection_process_new_packets'
/usr/bin/ld: CMakeFiles/client.dir/client.c.o: in function `send_request_and_read_response':
client.c:(.text+0x416): undefined reference to `rustls_version'
/usr/bin/ld: client.c:(.text+0x4c6): undefined reference to `rustls_connection_write'
/usr/bin/ld: client.c:(.text+0x591): undefined reference to `rustls_connection_wants_read'
/usr/bin/ld: client.c:(.text+0x625): undefined reference to `rustls_connection_wants_write'
/usr/bin/ld: client.c:(.text+0x67a): undefined reference to `rustls_connection_wants_read'
/usr/bin/ld: client.c:(.text+0x690): undefined reference to `rustls_connection_wants_write'
/usr/bin/ld: CMakeFiles/client.dir/client.c.o: in function `do_request':
client.c:(.text+0xb79): undefined reference to `rustls_client_connection_new'
/usr/bin/ld: client.c:(.text+0xc0c): undefined reference to `rustls_connection_set_userdata'
/usr/bin/ld: client.c:(.text+0xc22): undefined reference to `rustls_connection_set_log_callback'
/usr/bin/ld: client.c:(.text+0xc62): undefined reference to `rustls_connection_free'
/usr/bin/ld: CMakeFiles/client.dir/client.c.o: in function `verify':
client.c:(.text+0xcef): undefined reference to `rustls_slice_slice_bytes_len'
/usr/bin/ld: client.c:(.text+0xd93): undefined reference to `rustls_slice_slice_bytes_get'
/usr/bin/ld: CMakeFiles/client.dir/client.c.o: in function `main':
client.c:(.text+0xed7): undefined reference to `rustls_client_config_builder_new'
/usr/bin/ld: client.c:(.text+0xf2b): undefined reference to `rustls_client_config_builder_load_roots_from_file'
/usr/bin/ld: client.c:(.text+0xf84): undefined reference to `rustls_client_config_builder_dangerous_set_certificate_verifier'
/usr/bin/ld: client.c:(.text+0xfc3): undefined reference to `rustls_client_config_builder_set_alpn_protocols'
/usr/bin/ld: client.c:(.text+0xfcf): undefined reference to `rustls_client_config_builder_build'
/usr/bin/ld: client.c:(.text+0x101d): undefined reference to `rustls_client_config_free'
/usr/bin/ld: CMakeFiles/client.dir/common.c.o: in function `print_error':
common.c:(.text+0x51): undefined reference to `rustls_error'
/usr/bin/ld: CMakeFiles/client.dir/common.c.o: in function `write_tls':
common.c:(.text+0x2c5): undefined reference to `rustls_connection_write_tls_vectored'
/usr/bin/ld: common.c:(.text+0x2e2): undefined reference to `rustls_connection_write_tls'
/usr/bin/ld: CMakeFiles/client.dir/common.c.o: in function `copy_plaintext_to_buffer':
common.c:(.text+0x523): undefined reference to `rustls_connection_read'
/usr/bin/ld: CMakeFiles/client.dir/common.c.o: in function `log_cb':
common.c:(.text+0x868): undefined reference to `rustls_log_level_str'
collect2: error: ld returned 1 exit status
gmake[2]: *** [tests/CMakeFiles/client.dir/build.make:113: tests/client] Error 1
gmake[2]: Leaving directory '/home/jsha/rustls-ffi/build'
gmake[1]: *** [CMakeFiles/Makefile2:128: tests/CMakeFiles/client.dir/all] Error 2
gmake[1]: Leaving directory '/home/jsha/rustls-ffi/build'
gmake: *** [Makefile:91: all] Error 2

Happy to do some digging here but if you spot the issue let me know.

jsha commented 2 years ago

Oh, of course, this is still # TODO on UNIX and APPLE, so not expected to work on those platforms; I misunderstood the PR description.

divergentdave commented 2 years ago

Yeah, passing --config Debug when using anything but MSVC will have no effect on the C build, but it'll trick the generator expressions into running the cargo debug build. Then, the C will still be defaulted to -O3 or what have you, it'll try to use the optimized library dependency, and that mismatch should cause that error.

divergentdave commented 2 years ago

That sounds good, I believe they have feature parity. I'll make the change and push it.

rustls / rustls-ffi

Add CMake build system (with Windows support) #253