Missing third dot in cabal output on Windows only on two tests

[jasagredo]

A couple of tests are failing with the following mismatch in the output:

-Preprocessing library for plain-0.1.0.0...
-Building library for plain-0.1.0.0...
-Preprocessing test suite 'test' for plain-0.1.0.0...
-Building test suite 'test' for plain-0.1.0.0...
+Preprocessing library for plain-0.1.0.0..
+Building library for plain-0.1.0.0..
+Preprocessing test suite 'test' for plain-0.1.0.0..
+Building test suite 'test' for plain-0.1.0.0..
 Running 1 test suites...
 Test suite test: RUNNING...
 Test suite test: PASS
-Test suite logged to: <ROOT>/cabal.dist/work/./dist/build/<ARCH>/ghc-<GHCVER>/plain-0.1.0.0/test/plain-0.1.0.0-test.log
-Package coverage report written to <ROOT>/cabal.dist/work/./dist/build/<ARCH>/ghc-<GHCVER>/plain-0.1.0.0/hpc/vanilla/html/hpc_index.html
+Test suite logged to: cabal.dist/work/./dist/build/<ARCH>/ghc-<GHCVER>/plain-0.1.0.0/test/plain-0.1.0.0-test.log
+Test coverage report written to cabal.dist/work/./dist/build/<ARCH>/ghc-<GHCVER>/plain-0.1.0.0/hpc/vanilla/html/test/hpc_index.html
 1 of 1 test suites (1 of 1 test cases) passed.
-Package coverage report written to <ROOT>/cabal.dist/work/./dist/build/<ARCH>/ghc-<GHCVER>/plain-0.1.0.0/hpc/vanilla/html/hpc_index.html
+Package coverage report written to cabal.dist/work/./dist/build/<ARCH>/ghc-<GHCVER>/plain-0.1.0.0/hpc/vanilla/html/plain-0.1.0.0/hpc_index.html

Notice that the lines at the beginning have two dots at the end instead of three. This is not an artifact of normalization, cabal does output that. I have no idea why this is the case.

The following tests are affected:

• PackageTests/Regression/T9640/cabal.test.hs
• PackageTests/CustomTestCoverage/cabal.test.hs

[jasagredo]

cc @alt-romes as it seems you were the one adding those tests?

[mpickering]

@jasagredo This is because you are running the tests with the boot version of Cabal and Cabal-syntax. (See --intree-cabal-lib argument to cabal-tests in validate.sh).

See 716b109c4a

commit 716b109c4ae908458b16af5d75c233c7d9fdfc06 (mpickering/wip/cabal-install-Cabal)
Author: Matthew Pickering <matthewtpickering@gmail.com>
Date:   Tue Jan 30 12:19:22 2024 +0000

    Allow using different Cabal library versions for `cabal-install` tests with custom setup.

    The idea here is to pass a `--package-db` flag to `cabal-install` which
    contains just `Cabal` and `Cabal-syntax` of the specific version. This
    allows `cabal-install` tests to use the in-tree `Cabal` version,
    something which you can easily run into and get very confused about when
    writing tests.

    There are a few options which can be passed to `cabal-tests` executable
    to control which Cabal library you will test against.

    1. --boot-cabal-lib specifies to use the Cabal library bundled with the
       test compiler, this is the default and existing behaviour of the
       testsuite.
    2. --intree-cabal-lib=<root_dir> specifies to use Cabal and Cabal-syntax
       from a specific directory, and `--test-tmp` indicates where to put
       the package database they are built with.
    3. --specific-cabal-lib=<VERSION> specifies to use a specific Cabal
       version from hackage (ie 3.10.2.0) and installs the package database
       into --test-tmp=<DIR>

    The end result is that changes in the Cabal library can be tested with
    cabal-install tests in the testsuite.

    There have been a number of confusing issues with people writing tests
    for changes in the Cabal library which never ran because of
    cabal-install tests always used the boot Cabal library (see #9425
    for one).

    Fixes #9681

There should be CI jobs added that test that cabal-install works with the bundled version of Cabal library but then this requires a more complicated normalisation story to paper over the differences in Cabal versions (as you observe).

[jasagredo]

Sounds like a plausible cause, however I changed nothing in the validate script. How come it works on Ubuntu (i.e. using the non-boot version of Cabal) but it doesn't in Windows?

[mpickering]

@jasagredo It depends on GHC version, I have seen this exact failure myself on linux.

[jasagredo]

Let's close this one. It happens on custom built GHCs, but there doesn't seem to be a Cabal issue, just that the tests will depend on the bootstrap cabal, breaking stuff.