GaloisInc / saw-script

The SAW scripting language.
BSD 3-Clause "New" or "Revised" License
442 stars 63 forks source link

Compute Coverage CI failing #2114

Closed mccleeary-galois closed 2 months ago

mccleeary-galois commented 2 months ago

Summary

Compute coverage CI is failing due to some sort of version issue.

Here you can see that the compute-coverage.sh is grabbing multiple versions from the dist-newstyle for this loop here

hpc markup 
--destdir=hpc-html 
--hpcdir=dist-newstyle/build/x86_64-linux/ghc-9.4.8/saw-script-1.2.0.99/hpc 
--hpcdir=dist-newstyle/build/x86_64-linux/ghc-9.4.8/saw-script-1.1.0.99/hpc/vanilla/mix/integration_tests 
--hpcdir=dist-newstyle/build/x86_64-linux/ghc-9.4.8/saw-script-1.1.0.99/hpc/vanilla/mix/saw 
--hpcdir=dist-newstyle/build/x86_64-linux/ghc-9.4.8/saw-script-1.1.0.99/hpc/vanilla/mix/saw-script-1.1.0.99 all.tix

Note the mismatch version numbers.

Cabal versions were updated in this PR from 1.1.0.99 -> 1.2.0.99. Need to examine why compute-coverage.sh is grabbing both.

Acceptance Criteria

compute-coverage.sh no longer fails on PRs which bump cabal version numbers

Do

sauclovian-g commented 2 months ago

compute-coverage.sh just grabs everything in dist-newstyle, so it'll never work across a version bump without cleaning the old version.

(it doesn't quite grab everything, which is why the list above has only the top dir shown for 1.2.0.99; but anyway it ought to do something else to find the top hpc dir.)

anyway don't spend any more time chasing after this now, we can fix it later

kquick commented 2 months ago

This is what the CABAL_CACHE_VERSION is intended to handle. I bumped this in https://github.com/GaloisInc/saw-script/pull/2113/commits/d6b6062f3c1350b6b020f92da348320402a013a2 and the build is successful.

To be clear, the issue arises when the version number of the saw package is updated in its cabal file, not when the version of the cabal tool itself is changed. Trying to automate this would involve detecting the current version number in the cabal file and then updating the find in the CI to ignore other versions; this can be done, but it may be simpler just to document that the CABAL_CACHE_VERSION should be updated as part of the release process (and in the same or subsequent commit that changes the saw-script version in its cabal file).

RyanGlScott commented 2 months ago

Bumping CABAL_CACHE_VERSION does suffice as a workaround. That being said, it really shouldn't be necessary to do this. It should suffice to run find -name hpc in the specific subdirectory of dist-newstyle that contains the particular version of saw-script you care about.

Unfortunately, I'm not aware of a command that grabs this specific subdirectory. We can get pretty close, however. If we run the following command after building saw-script:

$ ghc-pkg --package-db dist-newstyle/packagedb/ghc-${GHC_VER} describe saw-script --simple-output

Then, among other output, we see:

import-dirs:
    /home/ryanscott/Documents/Hacking/Haskell/saw-script/dist-newstyle/build/x86_64-linux/ghc-9.4.8/saw-script-1.0.0.99/build

And that is almost the directory you need to search for .tix files (just replace .../build with .../hpc).

sauclovian-g commented 2 months ago

That's straightforward to handle with shellscript; I'll go fix it tonight

sauclovian-g commented 2 months ago

With that technique we still have to guess $GHC_VER. I think using cabal list-bin will work better, since the suffix on that (build/saw/saw) isn't going to change.