Bugs in expect_test test?

[liweijian]

It seems that there is a bug with bisect_ppx in our internal project, as some expect_tests are not being captured.

However, when I change some of the expected test case, make it trigger some diff, such as changing from 3 to 4:

And run the following command:

dune runtest --instrument-with bisect_ppx --force
bisect-ppx-report html
open _coverage/index.html

And it just been covered!

Just wondering if anything I'd missed?

PS

In order to verify if the bisect_ppx has the expect_test test bug, I have created a demo repository that basically mirrors the our internal project structure. It appears to work in the expect_test cases. For example:

❯ tree cl
cl
├── dune
├── in101.ml
└── test
    ├── dune
    └── intest.ml

Contents of cl/dune:

(library
 (name in101)
 (inline_tests)
 (preprocess
  (pps ppx_inline_test))
 (instrumentation
  (backend bisect_ppx)))

Contents of cl/in101.ml:

let rec fact n = if n = 1 then 1 else n * fact (n - 1)

Contents of cl/test/dune:

(library
 (name cltest101)
 (inline_tests)
 (libraries in101)
 (preprocess
  (pps ppx_expect))
 (instrumentation
  (backend bisect_ppx)))

Contents of cl/test/intest.ml:

let%expect_test _ =
  print_int (In101.fact 5);
  [%expect {| 120 |}]

By running the following commands, we can achieve 100% coverage:

❯ rm -rf _build;rm -rf _coverage
❯ dune runtest --instrument-with bisect_ppx --force
We come in peace.
❯ bisect-ppx-report html
❯ open _coverage/index.html

[aantron]

In order to verify if the bisect_ppx has the expect_test test bug, I have created a demo repository that basically mirrors the our internal project structure. It appears to work in the expect_test cases. For example:

Does this project reproduce the bug? From this text, I got the impression that when you tried to reproduce the bug in a smaller project, everything worked.

[mbarbin]

@liweijian I may have a related issue. Would you mind having a look at #411 and let me know if BISECT_FILE does anything for you? I'm preparing to use bisect_ppx more and would be grateful for any insights into the setup. Thanks!