Replace `assertions` by raising Errors

[skuschel]

I was reading through python exercise and It seems to me that the Defensive Programing session is introducing assertions, where I believe Type- or ValueErrors should be raised instead:

Assertions are used in algorithm development, ensuring a state, which might not be the obvious outcome or may be violated due to numerical errors (like ensuring, that the eigenvalues of a hermitean matrix are real after the matrix has been diagonlized). Throwing an assertion indicates, that something went wrong really really deep inside an algorithm and it is usually no way to fix this as a user. That is why compiler optimizations are typically removing assertions (which happens if a program is started with python -O). A function precheck should raise a TypeError or ValueErrror instead. This would have some advantages:

• This would be consistent with most python libraries.
• When students advance, they will -- at some point -- start to catch errors as well and they will try to catch TypeError and ValueError most often. An AssertionError should almost never be caught.
• The participants know various kinds of errors from the section before already. I find it more logical to understand that functions raise Exceptions and in the section after, I learn how to raise them myself.

For the post-checks it probably doesnt hurt to raise ValueErrors as well.

Regarding the test-driven development: I have done this in my own code and this is at least a few hours on its own. May it be more useful to introduce the try... except statement in that space? That would feel like a more coherent program to me. I dont really know how to approach this. Do you want me to open another issue for that discussion?

[annefou]

This is definitely an interesting discussion and I fully agree with you that introducing try... except would be useful. However, I am not sure it is important for novices. We usually do not have time to go through this section (Defensive programming). I think it would be very interesting to learn once you really start to code woth Python and are familiar enough to move to the next step.

[skuschel]

What do you think about the assertions? The try... except vs. test-driven development was just a side note. Maybe worth to open another issue for that?

[maxim-belkin]

Some time ago, I'd be strongly against adding try...except block to this lesson. However, I've recently heard that TDD (Test-Driven Development) doesn't provide any significant advantage over the standard approach in the long-run. The try...except block, on the other hand, is there to stay and will definitely be a useful tool for anyone. On yet another hand, we have to carefully consider the breadth of topics we cover. One of the responses that I got at SciPy 2017 was that we should really talk about argparse. And it's hard to disagree with that too...

[maxim-belkin]

And with regards to assertions... I tend to agree. Assertions are designed for lower level stuff and shouldn't be used as a replacement for regular if/else checks. Would you be willing to map out a replacement episode on try...except, @skuschel?

[annefou]

That would be awesome!

[servilla]

Note: this comment is in need to satisfy the Carpentries Checkout Procedure Part 1.

I believe that the section on "Defensive Programming" is off target. Defensive programming should be viewed as making your software resilent to errors that result from incorrect data provided to the software (whether direct from a user or otherwise), not necessarily the reliability of the software to respond correctly to a user based on embedded logic; this later concept is perhap more the domain of test-driven development.

This section currently emphasizes the use of assertions, which does not improve resiliency - asserts, in general, result in catastrophic exceptions (i.e., the program aborts with a stack trace due to an incompatible state between what is expected and what is actually occurring) and are generally used in test driven development to indicate whether the software is performing as stated in a requirement (in other words, does the software perform to a given specifcation). The use of "assert" in production Python code is generally used for debugging purposes, thus the compile optimization flag indicated by @skuschel to ignore such code while during production operation. I agree with @skuschel that a better approach to this section would be to replace "assertions" with "raise" (exception), along with a "try/catch" scenario, which is better suited toward defensive programming. With a "try/except" pattern, the software can "catch" and respond to erroneous data without resorting to immediate termination (or if it does, then it can provide more meaninful and human consumbable information to correct the situation). This approach is also a more natural progression from the prior section on "Errors and Exceptions".

In summary, I don't believe that "test driven development" and "defensive programming" should be conflated as one and the same. As such, this section could be rewritten to represent "defensive programming" using Python's "try/except"; alternatively, it could be rewritten to address the practice of "test driven development", which may be beyond the scope of the Carpentries lessons.

[varioustoxins]

I think this has merit. Asserts really make more sense as a part of testing. Since we currently don’t cover testing in that much depth but do cover almost by definition programs with

1. user inputs
2. situations where exceptions can get thrown such as missing files or bad inputs to algorithms

I see exceptions as a good target as programmers need to familiar with them anyway...

It is also fairly easy to retain the defensive programming theme with exceptions while not commiting students to obscure errors in the future.

[crowegian]

Note: Also part of a Carpentries Checkout procedure.

I guess whether or not to teach defensive programming or test-driven development depends on the audience. Though I agree that both are important, time is an issue in these lessons so adding a new lesson to cover both separately is probably out of the question. I come from more of research background and my experience with the carpentries has mostly been in academia. Very few people are concerned with writing code that will handle user errors down the road, and more concerned with writing code that handles their narrow definition of the intended use case. So raising assertions to check that functions are running as intended, and variables look as they should be is the main concern of the developers in academia that I've seen. Once code is running as it should be, and they have results, then writing a defensive program usually comes to mind. I think academic participants in the carpentries would benefit from knowing best practices and resources to learn them on their own, while relying on this lesson to learn how to develop software which works as they intend it to.

I think keeping the lesson more or less as it is while either using asserts or try/except statements is good, and providing participants with the knowledge of how test-driven development and defensive programming differ (the latter for future use) would be helpful.

[davidbenncsiro]

Interesting discussion!

Teaching testing in general is of value, since it emphasises that a non-trivial part of the effort of software development is testing, not just coding. TDD brings this into sharp relief.

Assertions as a way of thinking about and encoding pre-conditions, post-conditions, and invariants, and so design by contract (DBC), are also valuable.

Some notion of assertions is found in any unit testing library/framework of course so there's at least some overlap.

I think that for new programmers, getting across the ideas of validation and verification (does the software meet the user's needs vs does the software meet the specification) is also valuable.

I agree with what others have said. If a way of distinguishing the use of inline assertions vs exceptions is necessary, I'd suggest that anything that fits into the DBC model, i.e. what is expected of functions/components in order for them to interact correctly, could emphasise assertions, while anything that relies upon user input or conditions that cannot be controlled by the way in which functions/components are composed by the programmer, should be reported via exception (or return value in some cases). The use of assertions in testing is orthogonal to this.

[bsmith89]

Relevant discussion in swcarpentry/curriculum-advisors#1.