search

Raku / ake

A Raku make-a-like inspired by rake
Artistic License 2.0
12 stars 10 forks source link

Outthentic tests for sake #26

Closed melezhik closed 5 years ago

melezhik commented 5 years ago

Hi! I played with sake a bit and I tried to port some existed features / closed bugs to Outthentic testing framework. The advantage of using the last - it is very easy to add more tests for new features/closed/existed bugs . It only requires Bash and Perl5 regexps. If it sounds interesting ? ... Open for discussion.

AlexDaniel commented 5 years ago

This is interesting, indeed. Thank you for your PR. I'll take a look again tomorrow. Having to create two files to just write a test is arguably less than awesome, but I'm not sure. Currently tests are written like this:

https://github.com/perl6/p6-sake/blob/08ce6c0628cd3332af726043c955b175d7b4b24a/t/23-deps.t#L9-L14

Which, in my opinion, is pretty good. You just give it a sakefile, test description, a command and what to test, and it works. Typically this takes just a few lines. stdout, stderr, exit code and signals are checked.

Maybe more complex scenarios can be tested with outthentic! I'll think about it.

AlexDaniel commented 5 years ago

Sorry, was busy for a few days. Looking at it again.

Here's a side by side comparison of the same test:

I fail to see the benefit. Not only it requires more files (which may end up messy once we get even more tests), but it also demands understanding of bash and outthentic.

Speaking of bash, cd $story_dir is better written as cd -- "$story_dir" (to avoid issues with spaces and dashes in paths). Perhaps it's not extremely important in this particular situation, but it is important in other cases, and my point here is that I'd much rather not involve bash if possible.

I'm leaning towards rejecting this, but I'll leave it open in case others want to comment.

melezhik commented 5 years ago

Hi! Thank you for commenting. Just my thoughts on your reply.

Bash is ok. There is nothing to learn here. Bash is simple and effective tool to create test environments from the scratch. Instead of running commands inside Perl6 using system/shell you can use native tool.

Bash matches good for the cli application tests.

The advantage with Bash it works fine with more complex cases, oneliner example mentioned here is not illustrative.

There is no mess with file structure in case of Outthentic. It is always clear model. You always have what you test separated from how you test. Tests always resides in separated folders. Sakefiles, check rules and Bash scenarios reside in different folders and files. Such a structure is much easier to support in scale in comparison when you mix everything in one file.

It is all IMHO, of-course. (;

melezhik commented 5 years ago

Closed due to dropping support of Outthentic - https://www.reddit.com/r/perl6/comments/ahbu54/sparrow_moves_to_perl6/