JUnit Jupiter Extension?

[julesjacobsen]

Have you considered creating a JUnit 5 extension so people can hook this directly into their unit tests?

https://junit.org/junit5/docs/current/user-guide/#extensions

[drseb]

No. Never heard of this. Would this work for different programming languages? The linked user guide seems pretty java-specific

[julesjacobsen]

No, it's just a JVM-based solution. But whatever python people use will need to have a separate implementation. Still - we only need these two.

The cool thing would be that in my code I'd be able to write a JUnit unit test like this:

@ExtendWith(PhenoUnitExtension.class)
@Test
void testIC(@TestSetDefinition IcTest[] icTests) {
    // ...
}

[drseb]

Not sure we're on the same page here. If we want this to be really independent of any programming language (there are also R-implementations), I feel that it makes a lot of sense to choose a human readable standard format, that is "easy" to process in all languages. I am no expert in python, but maybe something like this what you showed is also possible. Not sure. That's why I said that I need more input.

[julesjacobsen]

The tests can all be defined in yaml as you have already, but you need a language binding to expose that nicely for people to incorporate into their code in as native a way as possible. You've gone part-way with this solution already, this is a way to hook this into people's actual code.

The PhenoUnitTestingProvider.class you have would be the extension.

[julesjacobsen]

Also pytest supports extensions of some sort. Essentially the language-specific testing framework needs to collect all the test cases defined here in yaml and make them programmatically available to the code running the tests.