built-in unit testing

[zmiimz]

add unit testing feature directly into the language.

Missing built-in unit testing is unacceptable freedom for any modern programming language and the development of critical software. If we are positioning Fortran as something better than a formula translator, we need unit testing directly built-in to the language. Currently, there are some external modules/ unit testing frameworks for that purpose (see http://fortranwiki.org/fortran/show/Unit+testing+frameworks ), but implemented in completely different languages (Ruby, Perl, etc.), and/or with several limitations originated from the restrictions of Fortran language itself. Therefore, unit testing should be added ideally only after the implementation of "standard library", "exception handling" and m.b. after "generic/metaprogramming" features.

[everythingfunctional]

It depends on what you mean by "unit testing" and what you mean by "built-in". The definition I use for "unit testing" is executing a small "unit" of code (where small is a bit subjective), in isolation (i.e. it doesn't touch any external resources or any other pieces of production code), with known input(s), and verifying it produces the expected output(s) or behavior(s).

For me, I don't want my unit tests mixed in with my production code. I know there are some languages that enable, if not encourage that (Rust and maybe Python depending on the framework your using come to mind), but most languages I've used do not (Haskell, C++, Ruby, Javascript, Java, and many other's I haven't actually used).

The frameworks written in other languages are working around a limitation inherent in the kind of language that Fortran is, a compiled language without runtime reflection. Without that, you can't use Fortran to find the tests at run-time, you have to find them before compile time and construct a program that runs them. A sufficiently powerful preprocessor could make this doable "in Fortran", without having to invoke a separate tool prior to compilation, but that doesn't require any changes to the language itself (unless we want the standard to start defining prepocessor requirements too).

Do you have an example of the kind of code and test you'd like to write, but currently can't, and what specifically the language might need in order to make it work?

[zmiimz]

You can find a good implementation example here https://dlang.org/spec/unittest.html And yes, I suggest to have unit tests in the production code and to be able to run unit tests with every standard conforming compiler and without any external tool.

[milancurcic]

in isolation (i.e. it doesn't touch any external resources or any other pieces of production code), with known input(s)

I don't understand this. How do you test production code then? At some point you need to call a function or subroutine that is to be used in production and check its result, no? Usually, it's only the production code that I want to test.

[everythingfunctional]

I don't understand this. How do you test production code then? At some point you need to call a function or subroutine that is to be used in production and check its result, no? Usually, it's only the production code that I want to test.

Right, but you want to test only that one function or subroutine (unit) in a given unit test. If it wants to collaborate with some other resource (read from a file, print to screen, fire the missiles, etc.), you generally want to provide some sort of mock, stub, spy (whatever term you want to use) for it to use during the test. Otherwise it would be considered an integration test, since you're testing multiple units integrated together. Not that those kinds of tests aren't valuable as well, but they're not truly unit tests (at least by my definition).

[milancurcic]

Ah, I understand now, I agree. Indeed you wrote "any other pieces of production code". I should've read more carefully.

[everythingfunctional]

I suggest to have unit tests in the production code and to be able to run unit tests with every standard conforming compiler and without any external tool.

Is the idea then that the tests are run as part of the compilation process, but not included in the resulting executable? Or that the executable is able to "test itself" in some secondary mode of operation? I'm not familiar with D, so that example isn't particularly enlightening for me. Is D a compiled or interpreted language? What are the mechanics of running the unit tests?

[Leonard-Reuter]

@zmiimz I agree, that unit testing in the standard (like it exists for Python) would be perfect. Yet, for the time being, I can highly recommend pFUnit. PS: Many other modern languages do also not have a built-in unit test capability (e.g. C++)

[MarDiehl]

Rust and Julia, two rather recent languages, have direct support for unit testing. Since Fortran does not have (yet) a standard library it would be natural to have support for unit tests in the core language instead of relying on third-party libraries.

[certik]

(We are working on the standard library: stdlib.)