Fail to build on Windows

[alex-chung]

When I run bazel build //examples/dagger:coffee_app on Windows 10, it fails with the following error message: Error: Could not find or load main class org.jetbrains.kotlin.preloading.Preloader

I am running Bazel 0.10.1.

[hsyed]

This is likely down to path seperator issues. I’ll try my best to fix them but I do not have a windows machine to test with.

PR welcome :)

[hsyed]

It will be a while till I get around to this, waiting on the Bazel CI system to support windows. Also #40 prints pretty colours to the console -- need to find a suitable lib or something in Bazel core that abstracts console printing.

[buchgr]

@hsyed what's the blocker here? We do support Windows on Buildkite.

[hsyed]

@buchgr i’ve made a bit of progress. I am blocked on this issue.

[IljaKroonen]

@hsyed I'm trying to have a look at this but I have trouble connecting the dots. Is the https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/bazel/rules/java/java_stub_template_windows.txt file supposed to somehow "fix" the classpath when running a command? Is it not possible to check if we are on a windows or on a linux machine to change the separator in the piece of code you linked in the other thread? I suppose that would be a hack? https://github.com/bazelbuild/rules_kotlin/blob/740b962803b0a128832c4970587cc8d26dc80546/kotlin/internal/utils.bzl#L187-L187

[hsyed]

@IljaKroonen You are very brave !

If you are planning to work on the code I should push my current refactors. I've completely reorganized the skylark code.

There is an issue open in bazel core -- I couldn't figure out how to debug this particular script, it's like magic. I had to park the task and delete windows from virtualbox (80 gigs!).

It does more than just path replacements, it compensates for the lack of symlinks in windows.

I'll try to help you as best as I can but I recommend chasing up the issue in bazel core.

[hsyed]

@IljaKroonen ok I've just pushed my current refactor changes.

[IljaKroonen]

@hsyed I'm new to bazel and I'm having trouble navigating all of this. Until now I haven't really observed the java_stub_template.txt file not working - it seems it is used many times successfuly.

Instead, one thing I have trouble undestanding is how _fold_jars_action is supposed to work (or _maybe_make_srcsjar_action for that matter)

def _fold_jars_action(ctx, output_jar, input_jars):
    args=["--output", output_jar.path]
    for i in input_jars:
        args += ["--sources", i.path]

    ctx.action(
        mnemonic = "KotlinFoldOutput",
        inputs = input_jars,
        outputs = [output_jar],
        executable = ctx.executable._singlejar,
        arguments = args,
        progress_message="Merging Kotlin output jar " + output_jar.short_path
    )

In my environment, ctx.executable._singlejar is a jar file. Bazel tries to use CreateProcessW with the jar as a target, which fails because jar files are not executables. I fail to see how that could work on Linux.

INFO: Analysed 6 targets (0 packages loaded).
INFO: Found 6 test targets...
ERROR: C:/users/ilja/rules_kotlin/tests/integrationtests/jvm/basic/BUILD:40:1: Merging Kotlin output jar tests/integrationtests/jvm/basic/test_embed_resources.jar failed (Exit -1). Note: Remote connection/protocol failed with: execution failed: bazel-singlejar_deploy.jar failed: error executing command
  cd C:/users/ilja/_bazel_ilja/v37ldlyr/execroot/io_bazel_rules_kotlin
external/bazel_tools/tools/jdk/singlejar/bazel-singlejar_deploy.jar --output bazel-out/x64_windows-fastbuild/bin/tests/integrationtests/jvm/basic/test_embed_resources.jar --sources bazel-out/x64_windows-fastbuild/bin/tests/integrationtests/jvm/basic/test_embed_resources-ktclass.jar --sources bazel-out/x64_windows-fastbuild/bin/tests/integrationtests/jvm/basic/test_embed_resources-resources.jar
Action failed to execute: java.io.IOException: ERROR: src/main/native/windows/processes-jni.cc(239): CreateProcessW("C:\users\ilja\_bazel_ilja\v37ldlyr\execroot\io_bazel_rules_kotlin\external\bazel_tools\tools\jdk\singlejar\bazel-singlejar_deploy.jar" --output bazel-out/x64_windows-fastbuild/bin/tests/integrationtests/jvm/basic/test_embed_resources.jar --sources bazel-out/x64_windows-fastbuild/bin/tests/integrationtests/jvm/basic/test_embed_resources-ktclass.jar --sources bazel-out/x64_windows-fastbuild/bin/tests/integrationtests/jvm/basic/test_embed_resources-resources.jar): %1 is not a valid Win32 application.

I'll be trying to setup a linux box to compare behavior.

EDIT: I hacked it to run to run the jar with java (https://github.com/IljaKroonen/rules_kotlin/commit/dbb69d857a93d1bc4e3657d68ef9ee6f7fa95631), I now see the problem with the template :)

[hsyed]

@alex-chung Cool ! Things have moved on quite a bit since the WIP pr.

I've been working on making the builder testable without the rules. Those tests are in here

If you want you could get these tests green on windows. Very little changes should be required for this, just ensuring that the right path separators are used when joining classpath is all that is required. We can merge this in and then look at the changes needed to the rules. WDYT ?

[IljaKroonen]

@hsyed Sounds good.

I noticed something while trying to hack your macro in bootstrap.bzl for windows: on windows, bazel tries to run the macro twice with the same NAME and in the same directory. This creates a race with the generation and deletion of ${NAME}_temp.jar across both runs. Running with --jobs 1 does no present the same problem. Again, I'm wondering why this would be a windows specific issue.

I'll take another look this evening.

EDIT: https://docs.bazel.build/versions/master/be/make-variables.html says temporary variables should be written to $(@D), which seems to fix this issue. The current error i'm encountering is:

$ bazel test all_tests
INFO: Build options have changed, discarding analysis cache.
INFO: Analysed 8 targets (88 packages loaded).
INFO: Found 8 test targets...
ERROR: C:/users/ilja/rules_kotlin/src/test/kotlin/io/bazel/kotlin/builder/BUILD:59:1: Building src/test/kotlin/io/bazel/kotlin/builder/KotlinBuilderJvmTest.jar (1 source file) failed (Exit 1): java.exe
failed: error executing command
  cd C:/users/ilja/_bazel_ilja/v37ldlyr/execroot/io_bazel_rules_kotlin
  SET LC_CTYPE=en_US.UTF-8
  external/local_jdk/bin/java.exe -Xbootclasspath/p:external/bazel_tools/third_party/java/jdk/langtools/javac-9+181-r4173-1.jar -jar external/bazel_tools/tools/jdk/JavaBuilder_deploy.jar @bazel-out/x64_windows-fastbuild/bin/src/test/kotlin/io/bazel/kotlin/builder/KotlinBuilderJvmTest.jar-2.params
src\test\kotlin\io\bazel\kotlin\builder\tasks\jvm\KotlinBuilderJvmTest.java:1: error: cannot access io.bazel.kotlin.builder.tasks.jvm
package io.bazel.kotlin.builder.tasks.jvm;
^
  /io/bazel/kotlin/builder/toolchain/KotlinToolchain_KotlincInvoker_Factory.class
INFO: Elapsed time: 37.383s, Critical Path: 34.45s
INFO: 32 processes: 27 local, 5 worker.
FAILED: Build did NOT complete successfully
//src/test/kotlin/io/bazel/kotlin:KotlinJvmBasicAssertionTest         NO STATUS
//src/test/kotlin/io/bazel/kotlin:KotlinJvmDaggerExampleTest          NO STATUS
//src/test/kotlin/io/bazel/kotlin:KotlinJvmFriendsVisibilityTest      NO STATUS
//src/test/kotlin/io/bazel/kotlin:KotlinJvmKaptAssertionTest          NO STATUS
//src/test/kotlin/io/bazel/kotlin/builder:JdepsParserTest             NO STATUS
//src/test/kotlin/io/bazel/kotlin/builder:KotlinBuilderJsTest         NO STATUS
//src/test/kotlin/io/bazel/kotlin/builder:SourceJarCreatorTest        NO STATUS

FAILED: Build did NOT complete successfully

[hsyed]

@IljaKroonen strange. I had the builder building correctly with the changes here

Give that version a try -- it looks more idiomatic compared to the current one.

[hsyed]

My new laptop arrived. I'll start the lengthy process of installing windows in virtualbox again.

[hsyed]

the temp jar issue is likely ocuring because the file is unknown to Bazel. I just create it in the genrule without declaring it. The temp jar is only needed because I am normalizing the jar created by the kotlin compiler. Normalization is not needed here so i'll remove it for now.

i'll cherry pick some of the changes from wip after I get an environment up. stay tuned.

[hsyed]

@buchgr @cushon I'm trying to cherry pick previous windows fixes into the main branch.

I am using bazel 0.16.1 and have tried both msys2 and cmd.

This is the target I want to build: //src/main/kotlin:builder this is the error:

z:\github.com\bazelbuild\rules_kotlin>bazel --output_user_root=c:\tmp\bazel build //src/main/kotlin:builder
ERROR: While resolving toolchains for target //src/main/kotlin:builder: invalid registered toolchain '@io_bazel_rules_kotlin//kotlin/internal:default_toolchain': no such package '@io_bazel_rules_kotlin//kotlin/internal': No WORKSPACE file found in C:/tmp/bazel/rrdud4n4/external/io_bazel_rules_kotlin
ERROR: Analysis of target '//src/main/kotlin:builder' failed; build aborted: invalid registered toolchain '@io_bazel_rules_kotlin//kotlin/internal:default_toolchain': no such package '@io_bazel_rules_kotlin//kotlin/internal': No WORKSPACE file found in C:/tmp/bazel/rrdud4n4/external/io_bazel_rules_kotlin
INFO: Elapsed time: 0.341s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)

I can't even do a fetch on a something in @com_github_jetbrains_kotlin

z:\github.com\bazelbuild\rules_kotlin>bazel --output_user_root=c:\tmp\bazel fetch @com_github_jetbrains_kotlin//:kotlin-preloader
ERROR: no such package '@com_github_jetbrains_kotlin//': no such package '@io_bazel_rules_kotlin//kotlin/internal/repositories': No WORKSPACE file found in C:/tmp/bazel/rrdud4n4/external/io_bazel_rules_kotlin

the workspace com_github_jetbrains_kotlin does depend on io_bazel_rules_kotlin for the *_import rules but not in a way that would cause a cycle.

Any idea what the problem might be ? I am going to try bazel 0.15.2.

[IljaKroonen]

@hsyed I don't think I ever saw this error, can you please give me a commit hash I can checkout to try and reproduce this?

[hsyed]

@IljaKroonen current master should give you the error

p.s., I don't have a jdk installed, but I don't think that is needed.

[IljaKroonen]

@hsyed On master, I get the following output for both commands:

bazel --output_user_root=c:\tmp\bazel fetch @com_github_jetbrains_kotlin//:kotlin-preloader -> success, no output

And when doing the other command:

C:\Users\iljak\rules_kotlin>bazel --output_user_root=c:\tmp\bazel build //src/main/kotlin:builder
ERROR: Skipping '//src/main/kotlin:builder': no such package 'src/main/kotlin': BUILD file not found on package path
WARNING: Target pattern parsing failed.
ERROR: no such package 'src/main/kotlin': BUILD file not found on package path
INFO: Elapsed time: 0.208s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)

I am on windows 1803. I also have symlinks enabled on my computer, could that make a difference? And of course as you mentionned, I got plenty of stuff installed on it, including a JDK.

Probably unrelated because I don't think it gets called here, but java is called directly from the template in kotlin/builder/bootstrap.bzl, which would I believe not work without a JRE in the path

[IljaKroonen]

Apologies, I was not using the latest version

I got this:

C:\Users\iljak\rules_kotlin>bazel --output_user_root=c:\tmp\bazel fetch @com_github_jetbrains_kotlin//:kotlin-preloader

C:\Users\iljak\rules_kotlin>bazel --output_user_root=c:\tmp\bazel build //src/main/kotlin:builder
INFO: Analysed target //src/main/kotlin:builder (0 packages loaded).
INFO: Found 1 target...
ERROR: C:/users/iljak/rules_kotlin/src/main/kotlin/BUILD:18:1: Executing genrule //src/main/kotlin:compiler_lib_jar failed (Exit 1): bash.exe failed: error executing command
  cd C:/tmp/bazel/myksme6o/execroot/io_bazel_rules_kotlin
  SET PATH=C:\Program Files\Git\usr\bin;C:\Program Files\Git\bin;C:\Python27\;C:\Python27\Scripts;C:\Program Files\Docker\Docker\Resources\bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\nodejs\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Go\bin;C:\opscode\chefdk\bin\;C:\Program Files (x86)\Yarn\bin\;C:\Program Files\Gource\cmd;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\Git\cmd;C:\Program Files\Microsoft VS Code\bin;C:\ProgramData\chocolatey\bin;C:\Program Files\Java\jdk1.8.0_181\bin;C:\Ruby25-x64\bin;C:\Users\iljak\AppData\Local\Programs\Python\Python36-32\Scripts\;C:\Users\iljak\AppData\Local\Programs\Python\Python36-32\;C:\Users\iljak\AppData\Local\Microsoft\WindowsApps;C:\Program Files\Microsoft VS Code\bin;C:\bin;C:\Users\iljak\AppData\Roaming\npm;C:\zip\zip-3.0-bin\bin;C:\Users\iljak\go\bin;C:\Users\iljak\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin;C:\Users\iljak\AppData\Local\Yarn\bin;
  C:/Program Files/Git/usr/bin/bash.exe -c source external/bazel_tools/tools/genrule/genrule-setup.sh;
KOTLIN_HOME=external/com_github_jetbrains_kotlin

function join_by { local IFS="$1"; shift; echo "$*"; }

NAME=compiler_lib
CP="$(join_by : external/com_github_jetbrains_kotlin/lib/kotlin-annotation-processing.jar external/com_github_jetbrains_kotlin/lib/kotlin-compiler.jar external/com_github_jetbrains_kotlin/lib/kotlin-script-runtime.jar)"
ARGS="-jvm-target 1.8"

java -Xmx256M -Xms32M -noverify   -cp ${KOTLIN_HOME}/lib/kotlin-preloader.jar org.jetbrains.kotlin.preloading.Preloader   -cp ${KOTLIN_HOME}/lib/kotlin-compiler.jar org.jetbrains.kotlin.cli.jvm.K2JVMCompiler   -cp ${CP} -d ${NAME}_temp.jar ${ARGS} src/main/kotlin/io/bazel/kotlin/compiler/BazelK2JVMCompiler.kt

external/bazel_tools/tools/jdk/singlejar/bazel-singlejar_deploy.jar     --normalize     --compression     --sources ${NAME}_temp.jar     --output bazel-out/x64_windows-fastbuild/genfiles/src/main/kotlin/compiler_lib.jar

rm ${NAME}_temp.jar
warning: classpath entry points to a non-existent location: external\com_github_jetbrains_kotlin\lib\kotlin-annotation-processing.jar:external\com_github_jetbrains_kotlin\lib\kotlin-compiler.jar:external\com_github_jetbrains_kotlin\lib\kotlin-script-runtime.jar
src\main\kotlin\io\bazel\kotlin\compiler\BazelK2JVMCompiler.kt:18:22: error: unresolved reference: kotlin
import org.jetbrains.kotlin.cli.common.ExitCode
                     ^
src\main\kotlin\io\bazel\kotlin\compiler\BazelK2JVMCompiler.kt:19:22: error: unresolved reference: kotlin
import org.jetbrains.kotlin.cli.common.arguments.K2JVMCompilerArguments
                     ^
src\main\kotlin\io\bazel\kotlin\compiler\BazelK2JVMCompiler.kt:20:22: error: unresolved reference: kotlin
import org.jetbrains.kotlin.cli.common.messages.MessageRenderer
                     ^
src\main\kotlin\io\bazel\kotlin\compiler\BazelK2JVMCompiler.kt:21:22: error: unresolved reference: kotlin
import org.jetbrains.kotlin.cli.common.messages.PrintingMessageCollector
                     ^
src\main\kotlin\io\bazel\kotlin\compiler\BazelK2JVMCompiler.kt:22:22: error: unresolved reference: kotlin
import org.jetbrains.kotlin.cli.jvm.K2JVMCompiler
                     ^
src\main\kotlin\io\bazel\kotlin\compiler\BazelK2JVMCompiler.kt:23:22: error: unresolved reference: kotlin
import org.jetbrains.kotlin.config.Services
                     ^
src\main\kotlin\io\bazel\kotlin\compiler\BazelK2JVMCompiler.kt:26:48: error: unresolved reference: K2JVMCompiler
class BazelK2JVMCompiler(private val delegate: K2JVMCompiler = K2JVMCompiler()) {
                                               ^
src\main\kotlin\io\bazel\kotlin\compiler\BazelK2JVMCompiler.kt:26:64: error: unresolved reference: K2JVMCompiler
class BazelK2JVMCompiler(private val delegate: K2JVMCompiler = K2JVMCompiler()) {
                                                               ^
src\main\kotlin\io\bazel\kotlin\compiler\BazelK2JVMCompiler.kt:27:54: error: unresolved reference: K2JVMCompilerArguments
    private fun createArgs(args: Array<out String>): K2JVMCompilerArguments {
                                                     ^
src\main\kotlin\io\bazel\kotlin\compiler\BazelK2JVMCompiler.kt:45:59: error: unresolved reference: it
            delegate.parseArguments(tally.toTypedArray(), it)
                                                          ^
src\main\kotlin\io\bazel\kotlin\compiler\BazelK2JVMCompiler.kt:47:17: error: unresolved reference: it
                it.friendPaths = friendsPaths
                ^
src\main\kotlin\io\bazel\kotlin\compiler\BazelK2JVMCompiler.kt:52:75: error: unresolved reference: ExitCode
    fun exec(errStream: java.io.PrintStream, vararg args: kotlin.String): ExitCode {
                                                                          ^
src\main\kotlin\io\bazel\kotlin\compiler\BazelK2JVMCompiler.kt:54:25: error: unresolved reference: PrintingMessageCollector
        val collector = PrintingMessageCollector(errStream, MessageRenderer.PLAIN_RELATIVE_PATHS, arguments.verbose)
                        ^
src\main\kotlin\io\bazel\kotlin\compiler\BazelK2JVMCompiler.kt:54:61: error: unresolved reference: MessageRenderer
        val collector = PrintingMessageCollector(errStream, MessageRenderer.PLAIN_RELATIVE_PATHS, arguments.verbose)
                                                            ^
src\main\kotlin\io\bazel\kotlin\compiler\BazelK2JVMCompiler.kt:55:41: error: unresolved reference: Services
        return delegate.exec(collector, Services.EMPTY, arguments)
                                        ^
Target //src/main/kotlin:builder failed to build
INFO: Elapsed time: 2.177s, Critical Path: 1.90s
INFO: 0 processes.
FAILED: Build did NOT complete successfully

[hsyed]

ok it's because I am trying to do this over a mapped virtualbox drive. sigh.

[IljaKroonen]

So, with the following changes: https://github.com/IljaKroonen/rules_kotlin/pull/1/files

I got to the point where I can start making the builder tests work on windows.

The list of issues I know of so far are:

• The java template does not use the runfiles (this was the error you were blocked on, and is the cause of the 3 tests that don't have a status, if I understand correctly)
• 3/4 of the builder tests fail (that's what I will start investigating when I get time, I first have to understand what the tests do precisely)
• Behavior is reproductible only with --jobs 1; if I don't specify it, I get various errors; this could be because some parts of the build write to the same files concurrently

 bazel test all_tests --jobs 1
INFO: Analysed 8 targets (0 packages loaded).
INFO: Found 8 test targets...
FAIL: //src/test/kotlin/io/bazel/kotlin/builder:KotlinBuilderJvmTest (see C:/users/iljak/_bazel_iljak/myksme6o/execroot/io_bazel_rules_kotlin/bazel-out/x64_windows-fastbuild/testlogs/src/test/kotlin/io/bazel/kotlin/builder/KotlinBuilderJvmTest/test.log)
INFO: From Testing //src/test/kotlin/io/bazel/kotlin/builder:KotlinBuilderJvmTest:
==================== Test output for //src/test/kotlin/io/bazel/kotlin/builder:KotlinBuilderJvmTest:
Exception in thread "main" java.lang.ExceptionInInitializerError
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:264)
        at com.google.testing.junit.runner.BazelTestRunner.getTestClass(BazelTestRunner.java:153)
        at com.google.testing.junit.runner.BazelTestRunner.runTestsInSuite(BazelTestRunner.java:127)
        at com.google.testing.junit.runner.BazelTestRunner.main(BazelTestRunner.java:82)
Caused by: java.lang.IllegalStateException: file did not exist: external\com_github_jetbrains_kotlin\lib\kotlin-compiler.jar
        at io.bazel.kotlin.builder.utils.IOUtils.verified(IOUtils.kt:80)
        at io.bazel.kotlin.builder.utils.IOUtils.resolveVerified(IOUtils.kt:75)
        at io.bazel.kotlin.builder.toolchain.KotlinToolchain$Companion.createClassLoader(KotlinToolchain.kt:52)
        at io.bazel.kotlin.builder.toolchain.KotlinToolchain$Companion.createToolchain(KotlinToolchain.kt:72)
        at io.bazel.kotlin.builder.toolchain.KotlinToolchain.createToolchain(KotlinToolchain.kt)
        at io.bazel.kotlin.builder.tasks.jvm.KotlinBuilderJvmTest.<clinit>(KotlinBuilderJvmTest.java:34)
        ... 5 more
================================================================================
FAIL: //src/test/kotlin/io/bazel/kotlin/builder:KotlinBuilderJsTest (see C:/users/iljak/_bazel_iljak/myksme6o/execroot/io_bazel_rules_kotlin/bazel-out/x64_windows-fastbuild/testlogs/src/test/kotlin/io/bazel/kotlin/builder/KotlinBuilderJsTest/test.log)
INFO: From Testing //src/test/kotlin/io/bazel/kotlin/builder:KotlinBuilderJsTest:
==================== Test output for //src/test/kotlin/io/bazel/kotlin/builder:KotlinBuilderJsTest:
Exception in thread "main" java.lang.ExceptionInInitializerError
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:264)
        at com.google.testing.junit.runner.BazelTestRunner.getTestClass(BazelTestRunner.java:153)
        at com.google.testing.junit.runner.BazelTestRunner.runTestsInSuite(BazelTestRunner.java:127)
        at com.google.testing.junit.runner.BazelTestRunner.main(BazelTestRunner.java:82)
Caused by: java.lang.IllegalStateException: file did not exist: external\com_github_jetbrains_kotlin\lib\kotlin-compiler.jar
        at io.bazel.kotlin.builder.utils.IOUtils.verified(IOUtils.kt:80)
        at io.bazel.kotlin.builder.utils.IOUtils.resolveVerified(IOUtils.kt:75)
        at io.bazel.kotlin.builder.toolchain.KotlinToolchain$Companion.createClassLoader(KotlinToolchain.kt:52)
        at io.bazel.kotlin.builder.toolchain.KotlinToolchain$Companion.createToolchain(KotlinToolchain.kt:72)
        at io.bazel.kotlin.builder.toolchain.KotlinToolchain.createToolchain(KotlinToolchain.kt)
        at io.bazel.kotlin.builder.tasks.js.KotlinBuilderJsTest.<clinit>(KotlinBuilderJsTest.java:21)
        ... 5 more
================================================================================
FAIL: //src/test/kotlin/io/bazel/kotlin/builder:JdepsParserTest (see C:/users/iljak/_bazel_iljak/myksme6o/execroot/io_bazel_rules_kotlin/bazel-out/x64_windows-fastbuild/testlogs/src/test/kotlin/io/bazel/kotlin/builder/JdepsParserTest/test.log)
INFO: From Testing //src/test/kotlin/io/bazel/kotlin/builder:JdepsParserTest:
==================== Test output for //src/test/kotlin/io/bazel/kotlin/builder:JdepsParserTest:
JUnit4 Test Runner
.E.E
Time: 0.078
There were 2 failures:
1) parseJDK8Format(io.bazel.kotlin.builder.tasks.jvm.JdepsParserTest)
java.lang.AssertionError: expected:<7> but was:<8>
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.failNotEquals(Assert.java:743)
        at org.junit.Assert.assertEquals(Assert.java:118)
        at org.junit.Assert.assertEquals(Assert.java:555)
        at org.junit.Assert.assertEquals(Assert.java:542)
        at io.bazel.kotlin.builder.tasks.jvm.JdepsParserTest.testWithFixture(JdepsParserTest.java:118)
        at io.bazel.kotlin.builder.tasks.jvm.JdepsParserTest.parseJDK8Format(JdepsParserTest.java:100)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
        at com.google.testing.junit.runner.internal.junit4.CancellableRequestFactory$CancellableRunner.run(CancellableRequestFactory.java:89)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:138)
        at com.google.testing.junit.runner.junit4.JUnit4Runner.run(JUnit4Runner.java:112)
        at com.google.testing.junit.runner.BazelTestRunner.runTestsInSuite(BazelTestRunner.java:144)
        at com.google.testing.junit.runner.BazelTestRunner.main(BazelTestRunner.java:82)
2) parseJDK9Format(io.bazel.kotlin.builder.tasks.jvm.JdepsParserTest)
java.lang.AssertionError: expected:<1> but was:<2>
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.failNotEquals(Assert.java:743)
        at org.junit.Assert.assertEquals(Assert.java:118)
        at org.junit.Assert.assertEquals(Assert.java:555)
        at org.junit.Assert.assertEquals(Assert.java:542)
        at io.bazel.kotlin.builder.tasks.jvm.JdepsParserTest.testWithFixture(JdepsParserTest.java:119)
        at io.bazel.kotlin.builder.tasks.jvm.JdepsParserTest.parseJDK9Format(JdepsParserTest.java:105)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
        at com.google.testing.junit.runner.internal.junit4.CancellableRequestFactory$CancellableRunner.run(CancellableRequestFactory.java:89)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:138)
        at com.google.testing.junit.runner.junit4.JUnit4Runner.run(JUnit4Runner.java:112)
        at com.google.testing.junit.runner.BazelTestRunner.runTestsInSuite(BazelTestRunner.java:144)
        at com.google.testing.junit.runner.BazelTestRunner.main(BazelTestRunner.java:82)

FAILURES!!!
Tests run: 2,  Failures: 2

BazelTestRunner exiting with a return value of 1
JVM shutdown hooks (if any) will run now.
The JVM will exit once they complete.

-- JVM shutdown starting at 2018-08-21 23:09:27 --

================================================================================
ERROR: C:/users/iljak/rules_kotlin/src/test/kotlin/io/bazel/kotlin/BUILD:19:1: Compiling Kotlin to JVM //src/test/kotlin/io/bazel/kotlin:assertion_test_case { kt: 1, java: 0, srcjars: 0 } failed: Worker process quit or closed its stdin stream when we tried to send a WorkRequest:

---8<---8<--- Exception details ---8<---8<---
java.io.IOException:
        at com.google.devtools.build.lib.windows.WindowsSubprocess.writeStream(WindowsSubprocess.java:252)
        at com.google.devtools.build.lib.windows.WindowsSubprocess.access$000(WindowsSubprocess.java:33)
        at com.google.devtools.build.lib.windows.WindowsSubprocess$ProcessOutputStream.write(WindowsSubprocess.java:52)
        at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.doFlush(CodedOutputStream.java:2985)
        at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.write(CodedOutputStream.java:2917)
        at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.writeLazy(CodedOutputStream.java:2936)
        at com.google.protobuf.ByteString$LiteralByteString.writeTo(ByteString.java:1337)
        at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.writeBytesNoTag(CodedOutputStream.java:2735)
        at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.writeBytes(CodedOutputStream.java:2708)
        at com.google.devtools.build.lib.worker.WorkerProtocol$Input.writeTo(WorkerProtocol.java:217)
        at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.writeMessageNoTag(CodedOutputStream.java:2783)
        at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.writeMessage(CodedOutputStream.java:2759)
        at com.google.devtools.build.lib.worker.WorkerProtocol$WorkRequest.writeTo(WorkerProtocol.java:975)
        at com.google.protobuf.AbstractMessageLite.writeDelimitedTo(AbstractMessageLite.java:98)
        at com.google.devtools.build.lib.worker.WorkerSpawnRunner.execInWorker(WorkerSpawnRunner.java:305)
        at com.google.devtools.build.lib.worker.WorkerSpawnRunner.actuallyExec(WorkerSpawnRunner.java:154)
        at com.google.devtools.build.lib.worker.WorkerSpawnRunner.exec(WorkerSpawnRunner.java:112)
        at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:95)
        at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:63)
        at com.google.devtools.build.lib.exec.SpawnActionContextMaps$ProxySpawnActionContext.exec(SpawnActionContextMaps.java:362)
        at com.google.devtools.build.lib.analysis.actions.SpawnAction.internalExecute(SpawnAction.java:287)
        at com.google.devtools.build.lib.analysis.actions.SpawnAction.execute(SpawnAction.java:294)
        at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.executeActionTask(SkyframeActionExecutor.java:978)
        at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.prepareScheduleExecuteAndCompleteAction(SkyframeActionExecutor.java:910)
        at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.access$900(SkyframeActionExecutor.java:120)
        at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.call(SkyframeActionExecutor.java:763)
        at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.call(SkyframeActionExecutor.java:718)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.executeAction(SkyframeActionExecutor.java:457)
        at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.checkCacheAndExecuteIfNeeded(ActionExecutionFunction.java:513)
        at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.compute(ActionExecutionFunction.java:227)
        at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:400)
        at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:355)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
---8<---8<--- End of exception details ---8<---8<---

---8<---8<--- Start of log, file at C:/users/iljak/_bazel_iljak/myksme6o/bazel-workers/worker-113-KotlinCompile.log ---8<---8<---
ERROR: IOException while trying to read log file:
java.io.FileNotFoundException: C:\users\iljak\_bazel_iljak\myksme6o\bazel-workers\worker-113-KotlinCompile.log (The process cannot access the file because it is being used by another process)
        at java.base/java.io.FileInputStream.open0(Native Method)
        at java.base/java.io.FileInputStream.open(Unknown Source)
        at java.base/java.io.FileInputStream.<init>(Unknown Source)
        at java.base/java.io.FileInputStream.<init>(Unknown Source)
        at com.google.devtools.build.lib.vfs.AbstractFileSystem$ProfiledFileInputStream.<init>(AbstractFileSystem.java:116)
        at com.google.devtools.build.lib.vfs.AbstractFileSystem.createFileInputStream(AbstractFileSystem.java:66)
        at com.google.devtools.build.lib.vfs.AbstractFileSystem.getInputStream(AbstractFileSystem.java:46)
        at com.google.devtools.build.lib.vfs.Path.getInputStream(Path.java:779)
        at com.google.devtools.build.lib.vfs.FileSystemUtils$1.openStream(FileSystemUtils.java:369)
        at com.google.common.io.ByteSource.read(ByteSource.java:284)
        at com.google.common.io.ByteSource$AsCharSource.read(ByteSource.java:466)
        at com.google.devtools.build.lib.vfs.FileSystemUtils.readContent(FileSystemUtils.java:836)
        at com.google.devtools.build.lib.worker.ErrorMessage$Builder.logFile(ErrorMessage.java:61)
        at com.google.devtools.build.lib.worker.WorkerSpawnRunner.execInWorker(WorkerSpawnRunner.java:313)
        at com.google.devtools.build.lib.worker.WorkerSpawnRunner.actuallyExec(WorkerSpawnRunner.java:154)
        at com.google.devtools.build.lib.worker.WorkerSpawnRunner.exec(WorkerSpawnRunner.java:112)
        at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:95)
        at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:63)
        at com.google.devtools.build.lib.exec.SpawnActionContextMaps$ProxySpawnActionContext.exec(SpawnActionContextMaps.java:362)
        at com.google.devtools.build.lib.analysis.actions.SpawnAction.internalExecute(SpawnAction.java:287)
        at com.google.devtools.build.lib.analysis.actions.SpawnAction.execute(SpawnAction.java:294)
        at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.executeActionTask(SkyframeActionExecutor.java:978)
        at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.prepareScheduleExecuteAndCompleteAction(SkyframeActionExecutor.java:910)
        at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.access$900(SkyframeActionExecutor.java:120)
        at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.call(SkyframeActionExecutor.java:763)
        at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.call(SkyframeActionExecutor.java:718)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.executeAction(SkyframeActionExecutor.java:457)
        at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.checkCacheAndExecuteIfNeeded(ActionExecutionFunction.java:513)
        at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.compute(ActionExecutionFunction.java:227)
        at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:400)
        at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:355)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
---8<---8<--- End of log ---8<---8<---
INFO: Elapsed time: 1.936s, Critical Path: 0.59s
INFO: 3 processes: 3 local.
FAILED: Build did NOT complete successfully
//src/test/kotlin/io/bazel/kotlin/builder:SourceJarCreatorTest  (cached) PASSED in 0.5s
//src/test/kotlin/io/bazel/kotlin:KotlinJvmBasicAssertionTest         NO STATUS
//src/test/kotlin/io/bazel/kotlin:KotlinJvmFriendsVisibilityTest      NO STATUS
//src/test/kotlin/io/bazel/kotlin:KotlinJvmKaptAssertionTest          NO STATUS
//src/test/kotlin/io/bazel/kotlin/builder:JdepsParserTest                FAILED in 0.6s
  C:/users/iljak/_bazel_iljak/myksme6o/execroot/io_bazel_rules_kotlin/bazel-out/x64_windows-fastbuild/testlogs/src/test/kotlin/io/bazel/kotlin/builder/JdepsParserTest/test.log
//src/test/kotlin/io/bazel/kotlin/builder:KotlinBuilderJsTest            FAILED in 0.4s
  C:/users/iljak/_bazel_iljak/myksme6o/execroot/io_bazel_rules_kotlin/bazel-out/x64_windows-fastbuild/testlogs/src/test/kotlin/io/bazel/kotlin/builder/KotlinBuilderJsTest/test.log
//src/test/kotlin/io/bazel/kotlin/builder:KotlinBuilderJvmTest           FAILED in 0.5s
  C:/users/iljak/_bazel_iljak/myksme6o/execroot/io_bazel_rules_kotlin/bazel-out/x64_windows-fastbuild/testlogs/src/test/kotlin/io/bazel/kotlin/builder/KotlinBuilderJvmTest/test.log

FAILED: Build did NOT complete successfully

[IljaKroonen]

I started looking at JdepsParser. It looks like in this case, the main code is windows compatible, but not the test.

• The absolute path in the JDK8 fixture is not absolute on windows; I guess the fixture should have a windows absolute path there when the test runs on windows
• All paths in the fixtures are linux paths, and should be converted

I'll try to find some time tomorrow to fix the test.

[hsyed]

@IljaKroonen I've fixed the tests, i'll integrate your changes to bootstrap.bzl and commit it tomorrow.

[hsyed]

• [ ] Single jar as used in the rules needs to indirect between the native version on unix and the JVM version in windows. I tried updating it (see below) but it complains about --deploy_manifes_lines in the jvm version.

def _fold_jars_action(ctx, rule_kind, output_jar, input_jars):
    """Set up an action to Fold the input jars into a normalized ouput jar."""
    args = ctx.actions.args()
    args.add_all([
        "--normalize",
        "--compression",
    ])
    args.add_all([
        "--deploy_manifest_lines",
        "Target-Label: %s" % str(ctx.label),
        "Injecting-Rule-Kind: %s" % rule_kind,
    ])
    args.add("--output", output_jar)
    args.add_all(input_jars, before_each = "--sources")
#    inputs, _, input_manifests = ctx.resolve_command(tools = [ctx.attr._singlejar])
    ctx.actions.run_shell(
        mnemonic = "KotlinFoldJars",
        inputs = input_jars,
        outputs = [output_jar],
        command = """
case "{sj}" in
    *.jar)
        {java} -jar {sj} $@
        ;;
    *)
        {sj} $@
        ;;
esac
""".format( java = ctx.executable._java.path, sj = ctx.executable._singlejar.path ),
        arguments = [args],
        progress_message = "Merging Kotlin output jar %s from %d inputs" % (ctx.label, len(input_jars)),
    )

• [ ] The java stub template needs to have it's classpath adusted, one adjustment is to remove external/ from all all external workspaces. Another potential one is adding the local workspace name to local dependencies. There might be a perscribed way of doing this. Resolutions won't work otherwise.

[hsyed]

@cushon what is the best way to execute singlejar in a portable way ? Single Jar JVM version seems to have different semantics -- I can't add stamp lines to the jar (see above).

I used [this] approach for the bootstrap scripts.