kbilsted / StatePrinter

Automating unit testing and ToString() coding
Apache License 2.0
93 stars 32 forks source link

Custom Ordering #64

Open OmegaJak opened 2 years ago

OmegaJak commented 2 years ago

This PR implements #18. We are using StatePrinter with ApprovalTests for characterization tests that we run on our data model. Ordering of items sometimes changes in this model, but we don't want the tests to fail due to that most of the time. Custom orderers let us avoid that.

Thanks for the awesome library @kbilsted!

kbilsted commented 2 years ago

hi @OmegaJak

many thanks for the pr! Great work. Have glanced it and it looks good. Will merge in a few days.

Perhaps I can ask a favor of you? Would you be so kind as to check out https://github.com/kbilsted/ReassureTest.Net this is where I currently spend my energy, Perhaps you have opinions on that approach and whether it is better or worse for your current use-scenario.

OmegaJak commented 2 years ago

Happy to help!

I took a look at ReassureTest.Net, but I didn't see a way to use it with ApprovalTests. Here's how we're using StatePrinter and ApprovalTests:

We have files that we either designed to test persistence & loading or that we received from customers. We load these files into our application/library, then we run StatePrinter (heavily configured) on our top-level object containing the entire loaded model. We then give the string from StatePrinter to ApprovalTests, which compares the version it was just given to the one we have previously 'approved'. These text files are sometimes ~100,000 lines long.

Our goal with all this is to catch unexpected changes that somehow slipped past our thousands of more traditional unit/integration tests. As far as I can see, it doesn't look like ReassureTest.Net is as well suited for that scenario. It looks to me like it's trying to replace the more traditional unit tests. For that purpose, we have some tests that it might improve, but many that I don't think it would. Most of listed problems it's trying to solve are not not big problems for us, or have already addressed otherwise.

If I missed something about ReassureTest and you think it could work for us, I'd be happy to take another look! It seems like a really interesting new way of writing tests!