feature request: `runtests @class`

[cbm755]

I'd like to see runtests be the main driver. So runtests @myclass would run the tests in class @myclass.

Consider looking at Doctest: https://github.com/catch22/octave-doctest/blob/master/inst/doctest.m There we wanted doctest thing to DTRT:

• doctest my_fcn works
• doctest @klass works
• doctest octfile.oct works (and is subtly different than doctest octfile.
• doctest dir works recursively, possibly staying out of private/ subdirs (I forgot).

[mtmiller]

Why not test @myclass?

[cbm755]

A few years ago, I posted some patches to do some of this:

https://savannah.gnu.org/patch/?9220

Feel free to cherry pick if these still apply.

[mtmiller]

Is there a reason why @-sign would be required or preferable for this? Or would test myclass be sufficient?

[cbm755]

Good question... doctest seems to usually do the same thing for doctest @myclass and doctest myclass..., although I checked the code and their are some subtleties, e.g., doctest @logical (Symbolic has @logical/isAlways) methods and doctest logical (the builtin)

>> pkg load symbolic
>> pkg load doctest
>> doctest logical
Doctest v0.6.1: this is Free Software without warranty, see source.

logical ................................................ NO TESTS

Summary:

   PASS    0/0   

1/1 targets passed, 1 without tests.

>> doctest @logical
Doctest v0.6.1: this is Free Software without warranty, see source.

logical ................................................ NO TESTS
@logical/isAlways ...................................... PASS    2/2   

Summary:

   PASS    2/2   

2/2 targets passed, 1 without tests.

I think the same distinction could apply for #! BISTs.

[apjanke]

Is there a reason why @-sign would be required or preferable for this? Or would test myclass be sufficient?

You might need it to disambiguate between running tests for the whole class, and running tests for just the constructor. If you have a class named foo, then the function foo is the constructor for it, and matches the test <function_name> calling form, too.

[mtmiller]

You might need it to disambiguate between running tests for the whole class, and running tests for just the constructor.

Yeah, there was a similar discussion over at https://github.com/catch22/octave-doctest/issues/196, you might be interested in weighing in over there too.

[cbm755]

I posted a summary of my thinking based on that thread here

I think the same logic would apply for regular tests. Which suggests there could be some "parse this object for tests" common code.

[apjanke]

I think I mostly agree, and think that the whole test selection logic (and command syntax) for doctest and this BIST stuff can probably be the same.

My current thinking is that there are two layers of test discovery:

1. The lower layer that operates on files, using files to identify tests, and a parser that reads a source file and extracts N tests from it.
2. A higher user-facing layer that takes object names, directories, or other specifiers, and maps them to sets of files to draw tests from.
3. A command syntax layer on top of that that interprets the string arguments passed in to the test or runtests command-style functions and parses them into the specifier data structures used by layer 2.

But then if you wanted to test just a single method or constructor from a class, you couldn't use the specifier -> files -> tests mapping, because you'd only want some selected tests from within a classdef file. So that's probably not the right design after all.

[apjanke]

Oh – can BISTs actually be associated with a particular method within a classdef file? All the ones I recall seeing just have all the tests in a block down at the bottom of the file.

[cbm755]

I think almost everything about Octave's classdef support is in flux. What is certainly true is that separate .m files specifying class methods can have their own tests associated with them. That is @klasdef/mymethod.m as opposed to methods defined "inline" in @klasdef/klasdef.m.

[apjanke]

Ah, that makes sense. And that way, the 1 specifier -> 1 or more files -> all tests from those files mapping sequence works.

[apjanke]

This is in. It's in runtests2_refactor. You can call it with either of these forms:

runtests2_refactor @myclass
runtests2_refactor -class myclass

You can also do runtests2_refactor myclass, and it might work, depending on what else "myclass" could resolve to.

Support for namespaces and dot-referencing of constructors or class methods is not supported yet.

Why not test @myclass?

As things stand now, test acts on a single file, and classes may have multiple files in their definition. Unless we merge runtests and test, then the support needs to go at the runtests level.

[apjanke]

Closing this as fixed since the basic mechanism is working. Adding a #49 for adding namespace support.

[apjanke]

Ha! I found a quick-and-dirty way to add namespace support. https://github.com/apjanke/octave-testify/commit/39905642604642dc79947e38d56be7bfbf3e51c7