MrAnderson carries out integration tests against cider-nrepl and refactor-nrepl.
I think this is great.
But maybe we could tweak this a bit?
Use fetched sources of official release instead of git modules
Instead of including cider-nrepl and refactor-nrepl as git modules, fetch an official release's source and test against that. This would:
move this integration testing work under target and avoid tooling (like editors for example) picking up the git modules sources.
make it clearer exactly what official release we are testing against
I take this strategy for rewrite-clj canary testing against libs that use rewrite-clj and it seems to work nicely.
Consider comparing against an expected result
A nice diff of generated sources against an expected result would be helpful.
A developer could clearly see the effect of a MrAnderson change on generated sources.
If the change is deliberate, the expected result would be updated to the generated sources and committed back to the MrAnderson repo.
If the change is not deliberate, well, that would be certainly nice to know.
Expected results would have to updated any time we bumped versions of cider-nrepl and refactor-nrepl.
This would also ensure that generated sources z for lib x version y are always the same.
I've noticed that this might not always be the case (metadata maps, for example, might sometimes generate maps with same content but with key/vals in different orders).
I've been experimenting with difftastic for similar purposes in local diff debugging and finding it rather nice.
Consider testing against more libraries?
The known number of libraries that use MrAnderson is small.
I suppose we could add more to our suite.
And parallelize testing on CI.
Consider integration testing with unresolved tree
If we can't find a library in the wild that is using the unresolved tree feature, maybe pick one that isn't and generate sources and compare against an expected result for it but using the unresolve tree feature.
(Or... if the unresolved tree feature is not being used by anybody, maybe turf the feature?)
Next Steps
I await your feedback on these ideas.
If you are not interested, that's fine.
If you are interested in some or all, I am happy to put in the work.
Currently
MrAnderson carries out integration tests against cider-nrepl and refactor-nrepl.
I think this is great.
But maybe we could tweak this a bit?
Use fetched sources of official release instead of git modules
Instead of including cider-nrepl and refactor-nrepl as git modules, fetch an official release's source and test against that. This would:
targetand avoid tooling (like editors for example) picking up the git modules sources.I take this strategy for rewrite-clj canary testing against libs that use rewrite-clj and it seems to work nicely.
Consider comparing against an expected result
A nice diff of generated sources against an expected result would be helpful.
A developer could clearly see the effect of a MrAnderson change on generated sources. If the change is deliberate, the expected result would be updated to the generated sources and committed back to the MrAnderson repo. If the change is not deliberate, well, that would be certainly nice to know.
Expected results would have to updated any time we bumped versions of cider-nrepl and refactor-nrepl.
This would also ensure that generated sources z for lib x version y are always the same. I've noticed that this might not always be the case (metadata maps, for example, might sometimes generate maps with same content but with key/vals in different orders).
I've been experimenting with difftastic for similar purposes in local diff debugging and finding it rather nice.
Consider testing against more libraries?
The known number of libraries that use MrAnderson is small. I suppose we could add more to our suite. And parallelize testing on CI.
Consider integration testing with unresolved tree
If we can't find a library in the wild that is using the unresolved tree feature, maybe pick one that isn't and generate sources and compare against an expected result for it but using the unresolve tree feature.
(Or... if the unresolved tree feature is not being used by anybody, maybe turf the feature?)
Next Steps
I await your feedback on these ideas. If you are not interested, that's fine. If you are interested in some or all, I am happy to put in the work.