Possibly remove unittest docs disclaimer

[MatthewScholefield]

Right now there's the following disclaimer at the top of the unittest module docs:

Instead of unittest.nim, please consider to use the testament tool which offers process isolation for your tests. Also when isMainModule: doAssert conditionHere is usually a much simpler solution for testing purposes.

I feel like:

1. The mention about testament should be removed unless it gets properly released (installable via Nimble) with docs
2. Mentioning isMainModule: doAssert at the top (along with the mention of testament) makes the module feel like it's deprecated or something. I think it'd be better to remove that or have a more detailed explanation presenting isMainModule: doAssert as an option alongside unittest.

Does anyone have any particular qualms with this?

Thanks

[Araq]

But testament ships with Nim, there is no need to "nimble install" it. Unittest has open bugs that nobody is working on and once it's fixed, it's still the wrong design anyway IMO. Process isolation and running tests in parallel are good things. Testament offers them.

[MatthewScholefield]

Ah, that makes sense. But, there still aren't any docs on it and as a result I've never heard of a single project that uses it beyond Nim itself. That's why I think we could just remove the mention of it now and then if it gets properly documented and people start using it it's as simple as dropping it back in with a link.

[ephetic]

FWiW, I'm new to Nim, I had exactly the reaction described above, and testament does not appear to have been installed with Nim (at least there is not a binary in /usr/local/Cellar/nim/1.2.6/bin). After 15 minutes trying to get Google to not return results about the Bible, I found myself here.

[dom96]

It would be useful to ensure that testament is shipped but also to write a bit more about the different ways that tests can be written in Nim, including the trade offs and advantages of each method.

[ringabout]

Now, testament has docs.

http://nim-lang.github.io/Nim/testament.html

[MatthewScholefield]

Hi, @Araq can you reopen this? Testament having docs doesn't resolve the issue of unittest docs being confusing. Also, I think this issue's importance is highlighted by this reddit post where someone independently came to the same conclusion I did above and gathered ~20 upvotes with multiple comments within a short period of time.

If I were given the go-ahead, I'd be happy to replace the unittest page with a simple guide on using testament (something like a streamlined version of this). And after using testament myself, I can confirm it's perfectly suitable for simple use cases as long as you know how it works (basically .nim files in tests/ and run testament p tests or testament cat .).

[Araq]

If I were given the go-ahead, I'd be happy to replace the unittest page with a simple guide on using testament (something like a streamlined version of this). And after using testament myself, I can confirm it's perfectly suitable for simple use cases as long as you know how it works (basically .nim files in tests/ and run testament p tests or testament cat .).

Sounds great, thanks!

[timotheecour]

I can confirm it's perfectly suitable for simple use cases

testament is not without its own share of bugs though, https://github.com/nim-lang/Nim/issues/16838 in particular prevents using it inside nimble test.

replace the unittest page with a simple guide on using testament

please make sure not to duplicate what's already written in https://nim-lang.github.io/Nim/testament.html; it's ok to link to it from unittest.nim though (and vice versal from testament.html to unittest.html)

[MatthewScholefield]

testament is not without its own share of bugs though, #16838 in particular prevents using it inside nimble test.

Hmm, good to know testament is still a little rough around the edges. My view is that it's still the best general method of writing tests so I think it still makes sense so advertise it as such, perhaps noting to report bugs appropriately.

please make sure not to duplicate what's already written in https://nim-lang.github.io/Nim/testament.html

Yes, definitely. I will link directly to there for more details on testament and provide this page as simply an overview of a recommended method for unit testing.

[ringabout]

Now the testament tool is linked at the docs of unittest module, it is good enough.

https://nim-lang.github.io/Nim/unittest.html#testament

[kaushalmodi]

Now the testament tool is linked at the docs of unittest module, it is good enough.

So should this issue be closed?

[MatthewScholefield]

@kaushalmodi The first part of the issue has been addressed but the second part still remains:

Mentioning isMainModule: doAssert at the top (along with the mention of testament) makes the [unittest] module feel like it's deprecated or something. I think it'd be better to remove that or have a more detailed explanation presenting isMainModule: doAssert as an option alongside unittest.

Basically, as an end user I want to know what I should do to write unit tests. I don't want to have three confusingly similar solutions vaguely mentioned to me and then have to choose one at random or invest even more time into figuring out if testament is too complicated or if when isMaonModule is too primitive.

Solutions to this are:

1. Pick and choose one of the 3 options as the "Recommended" approach and add a blurb about this (ie. "For writing unit tests we suggest using tool X because Y. However, if you'd like to use this tool (Z) continue reading below")
2. At the top write something short that describes very clearly that there are 3 options for unit testing and in what cases you should use each one.

Personally I think there's no reason we can't have just one tool that works for both simple and complex cases. I'd vote for just always suggesting testament and if it's too complicated for simple use cases (even though for me it's been fine) long term we can just keep improving testament to make it easier to use for small projects.