Half of the test suite fails because 'nats-server' process cannot be started on Mac and Windows (but works on Linux)

HenriNext commented 2 weeks ago

Observed behavior

When running 'gradle test' task, half of the tests fail with error:

SEVERE: %%% Failed to run [nats-server --config /var/folders/g0/j11s_mg529gfq10lyqnxsgl80000gn/T/nats_java_test7737451283960269566.conf -js] SEVERE: Cannot run program "nats-server": error=2, No such file or directory SEVERE: %%% Make sure that the nats-server is installed and in your PATH. SEVERE: %%% See https://github.com/nats-io/nats-server for information on installation

Expected behavior

Tests not failing.

Server and client version

Server (signed and notarized) 2.8.2 on macOS Server (as downloaded) 2.10.21 on macOS, Windows and Linux Client source from latest main branch (commit 27efeab5)

Host environment

macOS 13.10 (failing) Windows 11 (failing) PopOS 22.04 (working) All with Java 11

Steps to reproduce

download nats-server
copy to location that is in $PATH
start nats-server once and accept all warning dialogs on Mac and Windows
start nats-server again to verify that no more interactive warning dialogs are shown
clone this repo
cd to repo folder
execute ./gradlew build and observe half the tests failing
set environment variable "nats_server_path" to point to correct location
execute ./gradlew build and observe half the tests failing again
modify source code and add line "NatsServerRunner.setPreferredServerPath("/usr/local/bin/nats");" to constructor of class NatsTestServer
execute ./gradlew build and observe the tests passing

Diagnosis: Your code fails to read environment variables on macOS and Windows.

scottf commented 2 weeks ago

@HenriNext I develop on Windows 11 and have no problem running the test suite multiple times daily. I also just double checked my ubuntu machine and no problem there either. I do not have access to a Mac.

It's strange that every test that requires a server doesn't fail. I set a bad nats-server_path on windows and got this:

Failed to run [\progams\bin --config C:\Users\<myuser>\AppData\Local\Temp\nats_java_test5603773406283295305.conf]

The code that runs the server is found in the jnats-server-runner project. The nats-server_path needs the entire path to the executable, not just the directory, so expects something like:

/usr/local/bin/nats/nats-server

This should not work since it does not point to an executable so I would expect all tests taht require a server to fail. (There are many that don't, for instance tests that check builders and objects. This could account for "half")

NatsServerRunner.setPreferredServerPath("/usr/local/bin/nats");

Here is excerpted code from the jnats-server-runner project:

public static final String NATS_SERVER_PATH_ENV = "nats_server_path";
public static final String DEFAULT_NATS_SERVER = "nats-server";

public static String getResolvedServerPath() {
    String serverPath = NatsServerRunner.getPreferredServerPath();
    if (serverPath == null) {
        serverPath = System.getenv(NATS_SERVER_PATH_ENV);
        if (serverPath == null) {
            serverPath = DEFAULT_NATS_SERVER;
        }
    }
    return serverPath;
}

This is my windows batch file that I use to run the tests:

taskkill /F /IM nats-server.exe

rd /S /Q C:\Users\<myuser>\AppData\Local\Temp\nats
rd /S /Q C:\Users\<myuser>\AppData\Local\Temp\jetstream

del C:\Users\<myuser>\AppData\Local\Temp\nats_java_test*.conf
del C:\Users\<myuser>\AppData\Local\Temp\nats_net_test*.conf

cd \nats\nats.java
call gradlew clean build
taskkill /F /IM nats-server.exe

HenriNext commented 2 weeks ago

Thanks for your fast response.

"half the test suite" meant exactly the (about) half that requires a nats-server, so nothing weird or non-deterministic.
I typo-ed, and the actual working solution was to call 'NatsServerRunner.setPreferredServerPath("/usr/local/bin/nats-server")', not just "/nats".
The surface problem seems to be that on macOS and Windows, the 'nats-server' was not found even when it is on PATH. So the advice from the readme section "Building from the source", and from the test suite error message when nats-server cannot be found, simply didn't work.
The underlying problem may be that when I traced the execution on macOS, the System.getenv(...) call returned null. I'm not sure of the reason, could it be related to running via Gradle? But this might break both the "nats-server from path" and the "nats-server from environment variable" cases, leaving "nats-server from preferredServerPath" as the only working option.

scottf commented 2 weeks ago

I run on windows with the server in my path. I have also tested using nats_server_path. Here is an interesting discussion about System.getEnv(...) failing, it has to do with the shell or the application without the proper path information in the environment.

Sorry for your difficulties, maybe there could be something of issue on a Mac since I don't have one to test, but it works on windows and linux (ubuntu, aws linux). Could be something with gradle, but I'm using two different gradle versions across a couple projects that use the jnats-server-runner, it works fine.

P.S. I'm having a friend check the build on a Mac, he's always worked on a Mac and been a contributor, so I'm pretty sure the Mac issue would have surfaced by now.

nats-io / nats.java