Testing multiplatform code?

[krzema12]

Hi there!

I was wondering: is specnaz ever going to support testing of platform-agnostic code? As far as I see, now it's coupled with JVM platform (JUnit/TestNG). When one wants to test a platform-agnostic Kotlin code, I guess we're stuck with kotlin.test for now. I'd love to be able to write these tests conveniently with specnaz!

If you don't plan to support it, it would be nice to drop a sentence in the library description that it's solely for JVM.

Cheers!

[skinny85]

Hey Peter,

thanks for opening the issue. When you mean multiplatform code, do you specifically have Kotlin's JS support in mind?

Specnaz is actually structured into separate libraries. While there's a specnaz-kotlin-junit package, it only contains "glue" code that deals with integrating with JUnit. All of the logic is kept in a separate, testing library-agnostic module called specnaz (and specnaz-kotlin) - this is what allows Specnaz to support TestNG in addition to JUnit.

The only snag that I can potentially see with supporting JS is that the specnaz package that contains all of the core logic of the library is written in Java, not Kotlin. Do you know whether Kotlin's JS support includes some way of interacting with Java libraries?

[krzema12]

Multiplatform and JS - actually, Kotlin supports also native binaries, and maybe more platforms to come in the future. To better explain the vocabulary I use: when a project is created that is meant to target multiple patforms, some code can be made common for all or some platforms. I call such code platform-agnostic because it doesn't know about any platform it's compiled for (JVM, JS, linux's 64-bit binary, and so on). Also, such platform-agnostic code cannot be run on its own. Like an abstract class which cannot be instantiated, common Kotlin code needs to be ingested by some platform-specific project to be compiled/transpiled and run. To give you a concrete example what I'm working on, see my project https://github.com/krzema12/fsynth - the core directory contains the common code that doesn't know about JVM or JS. cli targets JVM and web targets JS. I even use kotlin.math.sin instead of java.lang.Math.sin because thanks to this, the common code is platform-agnostic.

Looking at the project structure of Specnaz, specnaz and specnaz-kotlin do abstract out the testing platform, but they are still coupled with the JVM. What I'd expect is to have another project - an even higher-level abstraction, this time over target platforms. So there could be a project called specnaz-common which could be used to write tests for the platform-agnostic, common code, and then platform-specific projects could depend on e.g. specnaz-jvm-junit and specnaz-js-mocha to actually run the tests for each platform. It's done the same way with built-in kotlin.test library: my core project depends on platform-agnostic: org.jetbrains.kotlin:kotlin-test-common and org.jetbrains.kotlin:kotlin-test-annotations-common, and then the two other projects depend on org.jetbrains.kotlin:kotlin-test, org.jetbrains.kotlin:kotlin-test-junit and org.jetbrains.kotlin:kotlin-test-js.

Addressing your last paragraph: using Java to write platform-agnostic library to test Kotlin code would be pain... So probably it would have to be written in pure, platform-agnostic Kotlin, i.e. depending only on kotlin-stdlib's parts meant for "common" (see cyan dots).

[skinny85]

Thanks for the detailed explanation @krzema12!

The reason I asked about the Java Kotlin multiplatform story is that the specnaz module doesn't have any dependencies other than the Java standard library. Which means it could also easily be written in pure Kotlin. But the issue is that such rewrite would have the consequence of pulling in the Kotlin standard library dependency for all JVM users of Specnaz, including those that write tests in Java. I don't think that's acceptable. Introducing a specnaz-common module doesn't really change that.

So, it seems like specnaz-common would have to be pretty much a duplicated version of specnaz, except written in Kotlin instead of Java. That kind of sucks...

Do you have any ideas on how can this problem be solved?

[krzema12]

Sure, Kotlin runtime would have to exist in all platform-specific versions of Specnaz, in JVM and also e. g. in JS. That's the cost of this additional layer of abstraction. But I personally wouldn't call it not acceptable. Specnaz is a test library, and as a user I wouldn't mind having a more versatile library for the cost of the slightly bigger test runtime. That what probably happens for kotlin.test library, I don't think we can do any better than this. Kotlin's runtime would be just one of Specnaz's dependencies, like Guava or Jackson libraries could be. Please note that avoiding using any external libraries can lead to reinventing the wheel.

Having duplicated code is of course out of the question, it's an operational pain. It would be certainly possible to avoid it (actually that's what can be avoided with the platform-independent code).

I glanced at other test libraries for Kotlin and they all target JVM. Maybe we should take a more in-depth look, and see if they plan to expand to multiplatform code, too.

[skinny85]

Perhaps my statement about adding the dependency being not acceptable was too strong :-). But I don't think it's free either. There is a reason that the Specnaz libraries don't depend on JUnit/TestNG/Kotlin's standard library currently. Also, the added versatility is irrelevant to Specnaz's Java users (which outnumber the Kotlin users), who would still pay the costs of the added dependency.

I actually had a different idea of how to make this work, without adding the extra dependency. I thought that we could transpile the Java code in specnaz, at build time, inside this new specnaz-common module, to Kotlin. Intellij IDEA has this feature built-in, so perhaps there is a way to programmatically invoke it somehow...? I'll open an issue to JetBrains, I'll see what they reply.

[krzema12]

I'm not sure if I understood you correctly: you'd like to transpile Java code to Kotlin platform-independent code, right? I don't think it's doable - IntelliJ transpiles Java to Kotlin for JVM. It's easy because each Java entity can be called from within Kotlin if targeting JVM. For Kotlin platform-independent code, the transpiler would have to know how to come up with JVM feature "foobar" in the standard Kotlin platform-agnostic library, and it's impossible since only a subset of platforms' features is covered in this common library. I hope I understood you correctly and I didn't mess something up.

I'm curious what JetBrains replies, maybe they'll propose yet another approach to the problem we're trying to solve here.

[skinny85]

Well, if it turns out you need JVM feature 'foobar' not present in the Kotlin platform-agnostic standard library to implement this specnaz-commons package, doesn't it simply mean Specnaz is too coupled to the JVM anyway, and you can't re-write it in platform-agnostic Kotlin at all? Why would transpiling be any different than writing it by hand for this issue?

[krzema12]

I meant a bit different thing. Let's assume you use java.lang.Math.max function in your Specnaz Java code. In Kotlin standard library, we have kotlin.math.max. Now, when using IntelliJ's feature to translate Java to Kotlin, IntelliJ would still use JVM's function instead of Kotlin's. In other words, even if it's possible to map Java to Kotlin 1:1, I don't think IntelliJ knows such mapping. Another example would be random API - before Kotlin 1.3, it didn't exist in the common library, although the Random class/feature in Java is not tightly coupled to JVM.

All of this may not be an issue in case of Specnaz, I'm just giving you a heads up what to expect if the approach proposed by you is chosen.

[krzema12]

For completeness, here are related issues in other testing frameworks for Kotlin:

• https://github.com/spekframework/spek/issues/327
• https://github.com/kotlintest/kotlintest/issues/242

[skinny85]

Issue opened to JetBrains: https://youtrack.jetbrains.com/issue/IDEA-202428

[skinny85]

I'm closing this one due to inactivity. If you still need anything related to this issue, please comment, and I'll be happy to re-open.

[krzema12]

It's fine, I've been using kotest for several years now for writing multiplatform tests and it works nice :)

[skinny85]

I assumed as much 😄 (although of course, I'm sad to lose you as a Specnaz customer 🙂).