sol / doctest

An implementation of Python's doctest for Haskell
https://hackage.haskell.org/package/doctest
MIT License
373 stars 73 forks source link

Error related to OverloadedLists not reported as such #389

Open ysangkok opened 2 years ago

ysangkok commented 2 years ago

Reproduction repo: https://github.com/ysangkok/openapi3/tree/janus/weird-doctest

 % cabal repl --with-ghc=doctest
Resolving dependencies...
Build profile: -w ghc-9.4.2 -O1
In order, the following will be built (use -v for more details):
 - openapi4-3.2.2 (exe:openapi4) (first run)
Configuring executable 'openapi4' for openapi4-3.2.2..
Preprocessing executable 'openapi4' for openapi4-3.2.2..
src/Data/UsesList.hs:4: failure in expression `import Data.UsesList'
expected: 
 but got: 
          ^
          <no location info>: error:
              Could not find module ‘Data.UsesList’
              It is not a module in the current program, or in any known package.

Examples: 3  Tried: 2  Errors: 0  Failures: 1
Error: cabal: repl failed for exe:openapi4 from openapi4-3.2.2.

If I remove :set -XOverloadedLists in src/Main.hs, the test passes.

It is confusing because there is no reporting hint that the error is caused by OverloadedLists.

andreasabel commented 1 year ago

I fetched the module from your reproduction repository:

module Data.UsesList () where

-- $setup
-- >>> import Data.UsesList

-- |
--
-- >>> inferParamSchemaTypes
-- []

inferParamSchemaTypes :: [Int]
inferParamSchemaTypes = (concat :: Foldable f => f [Int] -> [Int]) []

The $setup step should not be needed here, but I suppose it is needed in your application, right?

sol commented 1 year ago

My first thought was that this is somehow related to https://gitlab.haskell.org/ghc/ghc/-/issues/20670. However, I can't reproduce this with a regular cabal repl GHCi session. So not sure.

@andreasabel it doesn't really matter whether the $setup is needed, or not. If doctest behaves different from ghci then that's a bug by definition. We would at least want to understand why that is.

I'll not have time to work on this. If somebody feels inclined to investigate this, then I think you would want to understand what exactly doctest is doing differently from ghci and try to reproduce with a plain ghci session.

Ideally we would want to look at the --verbose output from doctest, but I think this is currently not possible, as we can't pass arguments to doctest via cabal repl. Maybe a wrapper script would work, not sure. Alternatively, for the purpose of testing, modify the source to always be verbose.

ysangkok commented 1 year ago

@andreasabel If I remove the import, I get:

          <interactive>:33:1: error:
              Variable not in scope: inferParamSchemaTypes

But I suppose you are right, it would be a more minimal test case without it, since this error is probably only shown because it fails to compile, even though it shouldn't. I just added the import because it wasn't totally clear to me whether the module was automatically imported into the doctest session.