search

gnzlbg / ctest

Automatic testing of FFI bindings for Rust
https://docs.rs/ctest
Apache License 2.0
122 stars 29 forks source link

Test C Macros with custom code #56

Closed asomers closed 5 years ago

asomers commented 5 years ago

C macros and inline functions generally have to be reimplemented in Rust. That's error-prone and leads to bugs like https://github.com/rust-lang/libc/issues/1210 and https://github.com/rust-lang/libc/issues/1190 . Could ctest support testing macro reimplementations with custom code? The general idea would be to compare the output of the C macro against the output of the Rust function, with the same inputs. Implementation would probably require wrapping each C macro with a wrapper that's callable from Rust.

alexcrichton commented 5 years ago

I think this is a case where unit tests should be used rather than this crate, there's not really any way to procedurally generate tests that are guaranteed to work for all macros.

asomers commented 5 years ago

But how would one create those unit tests? It would require writing custom C code. That's why this crate would be useful. My idea is that the user would still have to write the wrapping C functions by hand, but that this crate would take care of compiling them. Or is there an easier way?

alexcrichton commented 5 years ago

Oh well this crate relies on actually executing the target code to run the tests, so I'm imagining that #[test] would be used as normal in conjunction with custom-compiled C files

gnzlbg commented 5 years ago

@asomers I'm closing this, since this is now solved in libc via custom tests.

IIUC what you want is to mark a C function as a "C macro" and generate using the types of this function a C wrapper that calls the macro, and... then somehow test that they produce the same results (passing them some argument range in the rust side and checking the result)?

I think that, unless you have to do this very often, you might be better off writing the C wrappers by hand, and using quickcheck or similar on the Rust side to test these APIs. I think that this wouldn't be easy to add to ctest given how it is architected. The aim of this crate is to check that the Rust FFI ABI matches that of the C code being linked, but C macros have no ABI.

As in, I would suggest we close this as out-of-scope, but maybe could spawn a nice separate crate whose only job is to find C macros in a header file, and create wrapper functions or wrapper consts from a Rust FFI API, and automatically generate quickcheck tests in the Rust side. If that succeeds, maybe ctest could depend on that crate instead and re-export the functionality.

asomers commented 5 years ago

Yes, we should close it. What I did for libc is working fine. The only reason I opened an issue here was because I didn't realize how easy the cc crate was to use. I thought that I would need ctest just for its compiler support.