Document the Exercism testing philosophy

[parkerl]

Closes #8

I hope I have captured most of the discussion from the issue. Hopefully this is a decent start that can help drive the conversation on this forward.

Questions I have:

• Should this be focused only on test writing?
• Do we want to talk about test randomization here?
• Do we want to talk about test skipping here?
• Do we want to talk about other CI issues here?
• Do we need to re-iterate that only the languages core libraries should be used?
• Do we need to re-iterate that only the testing framework's core features should be used? Is this ever an issue?
• Have I emphasized the goal of small, manageable, TDD like steps enough?

[Insti]

Thanks for doing this @parkerl I think what you have here looks good.

We have a test philosophy section over at xruby:

Prioritize educational value over expert comprehension and make sure that things are clear to people who may not be familiar with Minitest and even Ruby.

Provide the information the student needs to derive the code to pass the test in a clear and consistent manner. Illustrate the purpose of the individual elements of the assertion by using meaningful variable names.

I'm a bit worried that this line:

Temporary variables should be avoided unless they add to the test's clarity.

Will encourage too few temporary variables. I find that I need to be constantly aware that I'm writing code that is going to be read by learners rather than lean, concise production code that is going to be read by already competent practitioners.

Your questions:

Do we want to talk about test randomization here? Do we want to talk about test skipping here? Do we want to talk about other CI issues here? Do we need to re-iterate that only the languages core libraries should be used? Do we need to re-iterate that only the testing framework's core features should be used? Is this ever an issue?

All these seem to be language/implementation specific, so I think we don't need to cover them here.

[kytrinyx]

Do we want to talk about test randomization here? Do we want to talk about test skipping here?

I think that it would be worth mentioning that many testing frameworks randomize in order to ensure that the tests are order-independent. This can mean that people get confusing and overwhelming test failure output when working on an exercise. For this reason, some language tracks use a directive (depends on the testing framework) to skip all but the first test, in order to ensure that the test failure simulates TDD. In these cases, people solving the exercise will need to remove the directive.

[kytrinyx]

I'm a bit worried that this line:

Temporary variables should be avoided unless they add to the test's clarity.

Will encourage too few temporary variables. I find that I need to be constantly aware that I'm writing code that is going to be read by learners rather than lean, concise production code that is going to be read by already competent practitioners.

This specifies unless they add to the test's clarity. I believe that's exactly what we're going for.

[kytrinyx]

I'm going to merge this and submit a small pull request that updates a couple of things that I think might be worth including.