Errors when trying to generate a test coverage report

meooow25 commented 7 months ago

I tried to generate a test coverage report and ran into an entire zoo of errors.
I'll describe the issues so we can try to get this fixed, and a workaround. Skip to the end for the workaround.

Issue 1

Let's start with

$ cabal test intmap-lazy-properties --enable-coverage
<...>
[ 5 of 37] Compiling Utils.Containers.Internal.PtrEquality ( <...> )

<no location info>: error:
    compiler/GHC/StgToCmm/Prim.hs:(344,32)-(345,74): Non-exhaustive patterns in lambda

Error: cabal: Failed to build containers-0.6.8 (which is required by
containers-tests-0).

The locations leads us to the emitPrimop for ReallyUnsafePtrEqualityOp. https://gitlab.haskell.org/ghc/ghc/-/blob/ghc-9.6/compiler/GHC/StgToCmm/Prim.hs?ref_type=heads#L344-345

I am baffled.

Let's continue with a workaround, by removing all the reallyUnsafePtrEquality# stuff in PtrEquality.hs.

Issue 2

$ cabal test intmap-lazy-properties --enable-coverage
<...>
Running 1 test suites...
Test suite intmap-lazy-properties: RUNNING...
Test suite intmap-lazy-properties: PASS
Test suite logged to:
/home/meooow/dev/containers/dist-newstyle/build/x86_64-linux/ghc-9.6.3/containers-tests-0/test/containers-tests-0-intmap-lazy-properties.log
hpc-ghc-9.6.3: can not find containers-0.6.8-inplace/Utils.Containers.Internal.StrictPair in ./.hpc, /home/meooow/dev/containers/dist-newstyle/build/x86_64-linux/ghc-9.6.3/containers-tests-0/hpc/vanilla/mix/containers-tests-0, /home/meooow/dev/containers/dist-newstyle/build/x86_64-linux/ghc-9.6.3/containers-tests-0/hpc/vanilla/mix/intmap-lazy-properties
CallStack (from HasCallStack):
  error, called at libraries/hpc/Trace/Hpc/Mix.hs:122:15 in hpc-0.6.2.0:Trace.Hpc.Mix
Error: cabal: Tests failed for containers-tests-0.

Okay we got the tests to run successfully. But hpc failed.
I find that I have two vanilla/mix dirs:

dist-newstyle/build/x86_64-linux/ghc-9.6.3/containers-tests-0/hpc/vanilla/mix
dist-newstyle/build/x86_64-linux/ghc-9.6.3/containers-0.6.8/hpc/vanilla/mix

For some reason, hpc attempts to find containers-0.6.8 mix files without being aware of the containers-0.6.8 directory. This looks like a Cabal issue? Let's try to work around it by running hpc manually.

Using hpc

I had to jump through a few hoops with hpc but they aren't really bugs so I won't go into the details. The gist is that I needed to go into containers-test and run hpc providing the tix file and mix dirs manually.

$ cd containers-tests
$ hpc markup \
  ../dist-newstyle/build/x86_64-linux/ghc-9.6.3/containers-tests-0/hpc/vanilla/tix/intmap-lazy-properties/intmap-lazy-properties.tix \
  --hpcdir ../dist-newstyle/build/x86_64-linux/ghc-9.6.3/containers-tests-0/hpc/vanilla/mix/containers-tests-0 \
  --hpcdir ../dist-newstyle/build/x86_64-linux/ghc-9.6.3/containers-tests-0/hpc/vanilla/mix/intmap-lazy-properties \
  --hpcdir ../dist-newstyle/build/x86_64-linux/ghc-9.6.3/containers-0.6.8/hpc/vanilla/mix/containers-0.6.8
Writing: containers-0.6.8-inplace/Utils.Containers.Internal.Coercions.hs.html
<...>
Writing: IntMapValidity.hs.html
Writing: hpc_index.html
Writing: hpc_index_fun.html
Writing: hpc_index_alt.html
Writing: hpc_index_exp.html

Success!

Workaround tl;dr

Delete the imports and usages of reallyUnsafePtrEquality# in PtrEquality.hs
Run the test with cabal test intmap-lazy-properties --enable-coverage. The test will run but hpc will fail, that's fine.
cd containers-tests

Run hpc providing the tix file and mix dirs

hpc markup \
  ../dist-newstyle/build/x86_64-linux/ghc-9.6.3/containers-tests-0/hpc/vanilla/tix/intmap-lazy-properties/intmap-lazy-properties.tix \
  --hpcdir ../dist-newstyle/build/x86_64-linux/ghc-9.6.3/containers-tests-0/hpc/vanilla/mix/containers-tests-0 \
  --hpcdir ../dist-newstyle/build/x86_64-linux/ghc-9.6.3/containers-tests-0/hpc/vanilla/mix/intmap-lazy-properties \
  --hpcdir ../dist-newstyle/build/x86_64-linux/ghc-9.6.3/containers-0.6.8/hpc/vanilla/mix/containers-0.6.8

treeowl commented 7 months ago

Of course, deleting those uses will prevent us from checking coverage of the relevant paths. Have you filed a GHC ticket? That first thing is an obvious compiler bug!

Bodigrim commented 7 months ago

СС @BinderDavid for hpc.

BinderDavid commented 7 months ago

This might have been fixed by https://github.com/haskell/cabal/pull/9464 . What is the version of cabal that has been used?

treeowl commented 7 months ago

This might have been fixed by haskell/cabal#9464 . What is the version of cabal that has been used?

This looks very different; it's complaining about an incomplete pattern match in the compiler when lowering STG to cmm.

clyring commented 7 months ago

Issue 1 is certainly a ghc bug. Do you want to file the upstream report? I think I know what's happening there; if I'm right I should have time to prepare and test a patch this weekend.

treeowl commented 7 months ago

I can probably do it sometime tonight, but I won't feel bad if someone beats me to the punch.

treeowl commented 7 months ago

@meooow25, issue 1 seems to be confined to the definition of hetPtrEq, which we don't actually use anywhere. So just deleting it should be a safe workaround.

treeowl commented 7 months ago

@clyring, I've filed the first issue upstream as #24608.

treeowl commented 7 months ago

@BinderDavid , ah, you were talking about the second issue. My bad.

meooow25 commented 7 months ago

@BinderDavid I'm using the latest Cabal 3.10.3.0

haskell / containers