search

alphapapa / makem.sh

Makefile-like script for linting and testing Emacs Lisp packages
GNU General Public License v3.0
163 stars 13 forks source link

WIP: Test suite #6

Open alphapapa opened 4 years ago

alphapapa commented 4 years ago

@DamienCassou Here's a WIP test suite. It sort of imitates Python by making makem.sh sourceable as a library, which allows its commands and functions to be called from inside the test suite, which simplifies testing them and reduces boilerplate:

https://github.com/alphapapa/makem.sh/blob/fcbaf8d4d7b89660c07f322a86246a380673f176/makem.sh#L866

Then I added an example project to test against, and there is code to test against a temporary, empty project for "should not" tests.

It seems to work pretty well so far.

However, I'm not sure this is all worth it. It feels very complicated. And to account for all the possible should-pass/should-not-pass scenarios would be lengthy and even more complicated. I feel like makem.sh is already well-tested by using it in my projects, and I always test it thoroughly when adding or changing code.

If you're interested enough to look at the test script, I'd be interested to know what you think. Thanks.

DamienCassou commented 4 years ago

Indeed, it seems the setup required to have unit-testing for makem.sh is huge. I think it's much more complex than for makel because of your choice to have makem.sh run for whole projects whereas makel can be configured to run on a single file.

If you don't really see the value of these tests it's probably better to stop spending time on that now :-). Thanks for trying

alphapapa commented 4 years ago

After further consideration, it doesn't seem that bad to me, and a few people now have expressed the desire for regression tests, so I may still merge this. It doesn't change makem.sh that much, so it seems not too unreasonable.

I think I won't necessarily write tests for every conceivable scenario immediately, though, as there are just so many possible combinations. But a few tests, and the ability to write regression tests as needed, seems like a good idea.

Does that sound reasonable to you? Thanks for your feedback.

DamienCassou commented 4 years ago

alphapapa notifications@github.com writes:

Does that sound reasonable to you? Thanks for your feedback.

I think that would be perfect! I suggest you add tests when you add features or fix bugs.

-- Damien Cassou

"Success is the ability to go from one failure to another without losing enthusiasm." --Winston Churchill