search

xd009642 / tarpaulin

A code coverage tool for Rust projects
https://crates.io/crates/cargo-tarpaulin
Apache License 2.0
2.5k stars 180 forks source link

Running coverage for non-cargo tests (end-to-end) #201

Closed D4nte closed 3 years ago

D4nte commented 5 years ago

Coming from C++ world with GoogleTest. I am used to be able to check code coverage from my "end-to-end" tests (ie, poking the binary from outside). It would usually go something like that

  1. compile binary with special flags to track coverage
  2. run end to end tests (this generate files on binary use)
  3. run gcov to report code coverage

However, as it looks like tarpaulin runs and check coverage for Rust tests, it does not seem possible.

Am I correct? Is there a way to run a tests suite and have tarpaulin report code coverage? If not, is that a feature you would consider?

xd009642 commented 5 years ago

You are correct, but it is a feature I would consider. It just adds extra complexity in specifying an appropriate build command and run command. It might be that representing tarpaulin configuration for a project via an external config file might be the best approach. I'll have a think about it.

On the other hand could restructure your testing approach to use rust integration tests to do this with tarpaulin as it currently is. For example, my main pretty much just parses the options to get the Config struct then passes that into a function called something like run_tarpaulin. I can then get end-to-end by just calling run_tarpaulin with an appropriate config struct.

D4nte commented 5 years ago

Yes, I absolutely considered using Rust as a harness to run the end to end tests and in this case it would be easy to use tarpaulin coverage. Thanks for flagging it as a potential enhancement.

xd009642 commented 3 years ago

So tarpaulin now lets you run binaries you can build via cargo build. Something like cargo tarpaulin --command build --bin $BIN_NAME -- ${ARGS}. This should be enough to get what you initially wanted if the binaries are built via cargo build (so there might be difficulties with cargo make). It also traces down into execed processes if they're part of the project (but this hasn't been released yet - but soon). So I'm going to close this as solved any issues if you comment on https://github.com/xd009642/tarpaulin/issues/507