search

taiki-e / cargo-llvm-cov

Cargo subcommand to easily use LLVM source-based code coverage (-C instrument-coverage).
Apache License 2.0
933 stars 57 forks source link

Support testing of binary crate #1

Closed taiki-e closed 3 years ago

taiki-e commented 3 years ago

Current status: https://github.com/taiki-e/cargo-llvm-cov/issues/1#issuecomment-855210584

taiki-e commented 3 years ago

On the library crate, it should work fine. 

$ git clone https://github.com/crossbeam-rs/crossbeam
$ cd crossbeam
$ cargo llvm-cov --all --exclude benchmarks --html
    Compiling...

...

Filename                                      Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
crossbeam-channel/src/channel.rs                  295                84    71.53%          61                 0   100.00%         411                97    76.40%
crossbeam-channel/src/context.rs                   59                13    77.97%          15                 0   100.00%         134                22    83.58%
crossbeam-channel/src/counter.rs                   27                 2    92.59%          11                 0   100.00%          75                 2    97.33%
crossbeam-channel/src/err.rs                      120               110     8.33%          12                 6    50.00%         117               111     5.13%
crossbeam-channel/src/flavors/array.rs            324               164    49.38%          37                 1    97.30%         810               405    50.00%
crossbeam-channel/src/flavors/at.rs                59                 3    94.92%          16                 0   100.00%          97                 5    94.85%
crossbeam-channel/src/flavors/list.rs             340               178    47.65%          36                 0   100.00%         803               407    49.32%
crossbeam-channel/src/flavors/never.rs              7                 0   100.00%           7                 0   100.00%          24                 0   100.00%
crossbeam-channel/src/flavors/tick.rs              46                 4    91.30%          15                 0   100.00%          85                 3    96.47%
crossbeam-channel/src/flavors/zero.rs             132                 2    98.48%          36                 0   100.00%         296                 3    98.99%
crossbeam-channel/src/lib.rs                        1                 0   100.00%           1                 0   100.00%           1                 0   100.00%
crossbeam-channel/src/select.rs                   248                39    84.27%          34                 1    97.06%         404                69    82.92%
crossbeam-channel/src/select_macro.rs               4                 0   100.00%           4                 0   100.00%           4                 0   100.00%
crossbeam-channel/src/utils.rs                     26                 3    88.46%           8                 0   100.00%          69                 2    97.10%
crossbeam-channel/src/waker.rs                     99                 1    98.99%          24                 0   100.00%         167                 0   100.00%
crossbeam-deque/src/deque.rs                      296                42    85.81%          34                 1    97.06%         998                66    93.39%
crossbeam-deque/src/lib.rs                          1                 0   100.00%           1                 0   100.00%           1                 0   100.00%
crossbeam-epoch/src/atomic.rs                     115                56    51.30%          46                 1    97.83%         440               237    46.14%
crossbeam-epoch/src/collector.rs                  155                 4    97.42%          33                 0   100.00%         280                12    95.71%
crossbeam-epoch/src/default.rs                     19                 0   100.00%          12                 0   100.00%          53                 0   100.00%
crossbeam-epoch/src/deferred.rs                    31                 1    96.77%          11                 0   100.00%          93                13    86.02%
crossbeam-epoch/src/epoch.rs                       20                 6    70.00%          13                 1    92.31%          56                 3    94.64%
crossbeam-epoch/src/guard.rs                       37                11    70.27%          11                 1    90.91%          59                21    64.41%
crossbeam-epoch/src/internal.rs                   149                20    86.58%          38                 2    94.74%         386                24    93.78%
crossbeam-epoch/src/lib.rs                          4                 0   100.00%           4                 0   100.00%          10                 0   100.00%
crossbeam-epoch/src/sync/list.rs                  105                 7    93.33%          19                 2    89.47%         250                 9    96.40%
crossbeam-epoch/src/sync/queue.rs                 216                 6    97.22%          41                 1    97.56%         338                 7    97.93%
crossbeam-queue/src/array_queue.rs                 58                 0   100.00%           9                 0   100.00%         202                 0   100.00%
crossbeam-queue/src/lib.rs                          1                 0   100.00%           1                 0   100.00%           1                 0   100.00%
crossbeam-queue/src/seg_queue.rs                   90                 6    93.33%          10                 0   100.00%         242                 9    96.28%
crossbeam-skiplist/src/base.rs                    434                61    85.94%          74                 0   100.00%        1071                85    92.06%
crossbeam-skiplist/src/lib.rs                       1                 0   100.00%           1                 0   100.00%           1                 0   100.00%
crossbeam-skiplist/src/map.rs                      12                 0   100.00%          12                 0   100.00%          60                 0   100.00%
crossbeam-skiplist/src/set.rs                       3                 0   100.00%           3                 0   100.00%          11                 0   100.00%
crossbeam-utils/src/atomic/atomic_cell.rs         246               193    21.54%          14                 1    92.86%         230                26    88.70%
crossbeam-utils/src/atomic/consume.rs               1                 0   100.00%           1                 0   100.00%           1                 0   100.00%
crossbeam-utils/src/atomic/seq_lock.rs             16                 3    81.25%           6                 0   100.00%          47                 5    89.36%
crossbeam-utils/src/backoff.rs                     19                 0   100.00%           4                 0   100.00%          41                 0   100.00%
crossbeam-utils/src/cache_padded.rs                 5                 0   100.00%           5                 0   100.00%          13                 0   100.00%
crossbeam-utils/src/lib.rs                          1                 0   100.00%           1                 0   100.00%           1                 0   100.00%
crossbeam-utils/src/sync/parker.rs                 47                 8    82.98%          14                 0   100.00%          99                 8    91.92%
crossbeam-utils/src/sync/sharded_lock.rs          143                88    38.46%          17                 2    88.24%         357               233    34.73%
crossbeam-utils/src/sync/wait_group.rs             13                 0   100.00%           5                 0   100.00%          40                 0   100.00%
crossbeam-utils/src/thread.rs                      34                 3    91.18%          14                 0   100.00%         132                 2    98.48%
src/lib.rs                                          1                 0   100.00%           1                 0   100.00%           1                 0   100.00%
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                            4060              1118    72.46%         772                20    97.41%        9011              1886    79.07%

Generated HTML

taiki-e commented 3 years ago

I think this is because this tool is only getting test coverage, but I'm not sure how I can fix it.

taiki-e commented 3 years ago

STATUS UPDATE: now work properly on the binary crate (the problem of incorrectly reporting as 100% coverage has been fixed), but I think we need to support cargo run.

$ git clone https://github.com/taiki-e/cargo-llvm-cov
$ cd cargo-llvm-cov
$ cargo llvm-cov
Filename                      Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
src/main.rs                       249               248     0.40%          38                37     2.63%         279               278     0.36%           0                 0         -
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                             249               248     0.40%          38                37     2.63%         279               278     0.36%           0                 0         -
taiki-e commented 3 years ago

Nah, what is really needed here is to actually support the results of tests that use the CARGO_BIN_EXE environment variable and others, not adds the run subcommand.