`cabal test` integration (GHC 9.4.5)

[benjaminweb]

Objective: How to let doctest run with cabal test? (I've spent considerable time to learn that I need to call cabal repl --with-ghc=doctest.)

Ideal outcome: README of doctest shows example of doctest integration in package.yaml or cabal file.

Steps to reproduce:

• checkout repo [1].
• run cabal test

> cabal test
Resolving dependencies...
Build profile: -w ghc-9.4.5 -O1
In order, the following will be built (use -v for more details):
 - dtest-0.1 (lib) (configuration changed)
 - dtest-0.1 (test:doctest) (dependency rebuilt)
Configuring library for dtest-0.1..
Preprocessing library for dtest-0.1..
Building library for dtest-0.1..
Preprocessing test suite 'doctest' for dtest-0.1..
Building test suite 'doctest' for dtest-0.1..
Running 1 test suites...
Test suite doctest: RUNNING...
src/Lib.hs:4: failure in expression `import qualified Data.HashMap.Strict as HM'
expected:
 but got:
          ^
          <no location info>: error:
              Could not find module ‘Data.HashMap.Strict’
              Perhaps you meant
                Data.Map.Strict (from containers-0.6.7)
                Data.IntMap.Strict (from containers-0.6.7)

Examples: 1  Tried: 1  Errors: 0  Failures: 1

Test suite doctest: FAIL
Test suite logged to:
/Users/benjamin/repos/dtest/dist-newstyle/build/aarch64-osx/ghc-9.4.5/dtest-0.1/t/doctest/test/dtest-0.1-doctest.log
0 of 1 test suites (0 of 1 test cases) passed.
Error: cabal: Tests failed for test:doctest from dtest-0.1.

Expected behaviour:

> cabal repl --with-ghc=doctest
Resolving dependencies...
Build profile: -w ghc-9.4.5 -O1
In order, the following will be built (use -v for more details):
 - dtest-0.1 (lib) (configuration changed)
Configuring library for dtest-0.1..
Preprocessing library for dtest-0.1..
Examples: 1  Tried: 1  Errors: 0  Failures: 0

[1] https://github.com/benjaminweb/doctest-cabal-integration

[sol]

I've spent considerable time to learn that I need to call cabal repl --with-ghc=doctest.

I think it's documented in the README. I'm not sure what else to do here, but I'm open to suggestions.

Ideal outcome: README of doctest shows example of doctest integration in package.yaml or cabal file.

The recommended way to run doctest is via cabal repl, so this is probably not going to happen.

However, as I mentioned before cabal exec cabal test should probably work for your example repo. Did you give it a try?

[benjaminweb]

It’s documented in the README. I was aware of that. —

My cabal test definition stems from a boilerplate I made some time ago for continuous integration. …one command and all tests are run: cabal test

If I can call it via cabal test (in the sense that it gets called like cabal repl), I assumed that it should work. I learned, I misassumed. The key is that the README didn’t tell me NOT to call it via cabal test for reason XYZ. => Do you think it’s valuable to add information to only call via cabal repl / not via cabal test?

Ideal outcome: README of doctest shows example of doctest integration in package.yaml or cabal file.

The recommended way to run doctest is via cabal repl, so this is probably not going to happen.

Just to confirm: There is no and there will be no cabal integration of doctest? What then is the recommended approach to run it along with other test suites with one simple command? If there is no, is the README making that explicitly clear?

However, as I mentioned before https://github.com/sol/doctest/issues/410#issuecomment-1671409548 cabal exec cabal test should probably work for your example repo. Did you give it a try?

Not yet, will do — once I am back at my machine.

[andreasabel]

@benjaminweb wrote:

Just to confirm: There is no and there will be no cabal integration of doctest?

You can try https://github.com/haskellari/cabal-doctest .

[benjaminweb]

@sol cabal run cabal test does not work for me. OverloadedStrings for example don't work. cabal repl --with-ghc=doctest takes 10 secs for 83 rather simple doctests. How can I speed this up for use with a filewatcher?

[benjaminweb]

@andreasabel got cabal-test now working (it is fast compared to cabal repl --with-ghc=doctest). Minimum working example: https://github.com/benjaminweb/doctest-cabal-integration/commit/2d781f701ff8b2b4e9bcb90badea2a5de935bc15 I've found the README frustrating, though. Adopting the simple-example was the only option that worked.

[andreasabel]

I've found the README frustrating, though. Adopting the simple-example was the only option that worked.

Yeah, I also just did cut-and-paste to get it working.
However, I didn't like to have a custom Setup.hs that is only needed for tests, possibly complicating installation for the regular users.
Thus, I now follow the cabal repl -w doctest path. This has the drawback that it needs custom CI, while I would love to just write cabal test.

[benjaminweb]

So, how is the speed compared to cabal-doctest in your case?

On Thu, 9 Nov 2023, at 18:29, Andreas Abel wrote:

I've found the README frustrating, though. Adopting the simple-example was the only option that worked.

Yeah, I also just did cut-and-paste to get it working. However, I didn't like to have a custom Setup.hs that is only needed for tests, possibly complicating installation for the regular users. Thus, I now follow the cabal repl -w doctest path. This has the drawback that it needs custom CI, while I would love to just write cabal test.

— Reply to this email directly, view it on GitHub https://github.com/sol/doctest/issues/411#issuecomment-1804259450, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ3EVP7GP3UG6PN5DVADYTYDUHIVAVCNFSM6AAAAAA3LFPZVOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBUGI2TSNBVGA. You are receiving this because you were mentioned.Message ID: @.***>