Architecture advice needed: automatically recognized tests mixed with source code

[Martinsos]

Hi, Thanks for this cool library :)!

I am setting up tasty as test framework for my project. From my previous experience in other languages (C++, JavaScript) I preferred having a setup where test files are next to the source files and not in the separate directory structure. So next to foobar.js I would put foobar.test.js . Also, test files would automatically get picked up by test runner based on their file extension and executed with the other tests. This is something I would like to achieve with Tasty however I am not sure how to approach it best.

From what I read so far about Tasty, I learned that tests have to be manually imported into main test file and incorporated into test tree. Also, you recommend putting them in separate directory structure.

I know this is question on somewhat higher lvl than Tasty itself since it also has to do with the how whole project is build, but was still hoping that you could give me some guidelines or suggestions on how to approach this, what is possible and what is not.

Also, if you think this is not a good approach, I would love to learn why.

My project is built with Stack.

Thanks a lot for any help!

[UnkindPartition]

Hi, thanks for the kind words!

Regarding the auto-discovery of tests, take a look at https://hackage.haskell.org/package/tasty-discover. However, I do not personally use that package, so I can't give you more specific advice. (Not that I have anything against that package/approach; as you say, habits play a big role here.)

As for placing the test and source files next to each other, I think you can do that (by specifying the same hs-source-dirs in the cabal file, or even omitting them altogether). The drawback of this approach (related to cabal/ghc, not tasty) is that, when you build the tests, the source files will be recompiled. So if you build both the tests and the library, the source files will be compiled twice. OTOH, if they are in separate directories, then the test component will simply link against the compiled library component. (At least this was the case several years ago; I don't know if any recent updates to cabal or ghc have changed that.)

Also, when I say cabal, I really mean Cabal-the-library, so it should apply equally to stack (which uses that library).

[Martinsos]

Thanks @feuerbach for the quick answer! I will take a look at tasty-discover, it seems like an interesting solution.

I did not know about that compile dependency, thanks for sharing that -> I will look into it, hopefully there is way to go around it.

Regarding the approach you are using, would you have a good example of a non-trivial project using that kind or similar kind of approach? I would love to better understand how it all works on the bigger scale.

Thanks!

[UnkindPartition]

Here are some of the projects that I've worked on (which aren't that big):

• https://github.com/haskell-suite/haskell-src-exts
• https://github.com/feuerbach/regex-applicative
• https://github.com/haskell-fswatch/hfsnotify

Some examples of much bigger projects and much bigger test suites:

• https://github.com/jgm/pandoc
• https://github.com/agda/agda
• https://github.com/idris-lang/Idris-dev
• git-annex (doesn't appear to have a web-browseable repo; you can clone it from git://git-annex.branchable.com/)

[Martinsos]

Awesome, this helps a lot! It might be worth adding a few of these real-life examples (like pandoc) to the README, so tasty users can get some inspiration / see what to expect, what do you think?

Thanks for help, I am closing the issue and will let you know if I manage to have test files in the src/.