Offer more (some!) approaches for template testing?

seancorfield / deps-new

Create new projects for the Clojure CLI / deps.edn

Eclipse Public License 1.0

352 stars 19 forks source link

Offer more (some!) approaches for template testing? #3

Open seancorfield opened 3 years ago

seancorfield commented 3 years ago

Dominic Monroe on Slack:

An approach to testing, even just a few basic utilities to spin up a repl of some kind and run some functions, testing that they don't explode would be great. Right now I have a bash script that generates a few variants of the project and then I manually check that they do sensible things (like that (dev) works, or that a server starts)

SevereOverfl0w commented 3 years ago

https://git.sr.ht/~severeoverfl0w/juxt-wedge/tree/master/item/test-template here's the script I use which generates multiple varieties. There's a bunch of things I would ideally test as I change things (e.g. after running (go) does an HTTP request to / work?).

I can appreciate that some of these might be out of scope for clj-new, but I expect they're also the most useful thing to test alongside a template. There're properties you want to have hold across a matrix of possible combinations of options.

seancorfield commented 3 years ago

Just so I understand, this is about some additional support for testing a new template as you are developing it specifically, rather than general project testing?

SevereOverfl0w commented 3 years ago

Yeah, I'd be interested in testing the template in some way. I suppose the tools I might need are:

A tool to generate a full matrix of all possible options
A way to run in the context of the generated projects, and run some kind of test suite in that context. Perhaps forking out a new JVM for each project and running a script that I provide that could do the testing itself.

SevereOverfl0w commented 2 years ago

I ended up with fairly simple tests.

Start JVM with pREPL
Run a few commands in that pREPL & check they do what I expect
Test the system actually came up (in this case by using etaoin to load a browser and take a picture)
Shut it all down cleanly

It would be nice to encourage template developers to do 1/2/4 by having something for this in the default template template. I rolled my own tools for that, I'm not sure if there's a programmatic client around for pREPL though.