It is impossible to use a custom wlp.usr.dir with arquillian

Hi,

Issue Overview

by any means, it is really complicated to use a custom wlp.usr.dir.

Expected Behaviour

Can set WLP_USER_DIR somewow in pom.xml and/or not use WLP_USER_DIR from environment.

Current Behaviour

When a user sets both WLP_USER_DIR in his personal environment and <userDirectory/> in liberty-maven-plugin is used, liberty-arquillian cannot be pointed easily to the correct directory.

Steps To Reproduce

Environment

WLP_USER_DIR=$HOME/.local/share/wlp-usr

`pom.xml`

Setting up liberty for arquillian

    <wlp.usr.dir>${project.build.directory}/wlp-usr</wlp.usr.dir>
    <liberty.env.wlp.usr.dir>${wlp.usr.dir}</liberty.env.wlp.usr.dir>

and the plugins


      <!-- configure wlp for arquillian -->
      <plugin>
        <groupId>io.openliberty.tools</groupId>
        <artifactId>liberty-maven-plugin</artifactId>
        <version>3.2</version>
        <extensions>true</extensions>
        <!-- Specify configuration, executions for liberty-maven-plugin -->
        <configuration>
          <serverName>ArquillianManagedServer</serverName>

          <assemblyArtifact>
            <groupId>io.openliberty</groupId>
            <artifactId>openliberty-runtime</artifactId>
            <version>[19.0.0.12,)</version>
            <type>zip</type>
          </assemblyArtifact>

          <serverXmlFile>src/test/liberty/config/server.xml</serverXmlFile>
          <userDirectory>${wlp.usr.dir}</userDirectory>

          <bootstrapProperties>
            <default.http.port>${testServerHttpPort}</default.http.port>
            <default.https.port>${testServerHttpsPort}</default.https.port>
          </bootstrapProperties>

          <skipTestServer>true</skipTestServer>
        </configuration>

        <executions>
          <execution>
            <id>create-server-management</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>create</goal>
            </goals>
          </execution>

          <execution>
            <id>configure-arquillian-xml</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>configure-arquillian</goal>
            </goals>
            <configuration>
              <arquillianProperties>
                <outputToConsole>true</outputToConsole>
              </arquillianProperties>
              <skipIfArquillianXmlExists>true</skipIfArquillianXmlExists>
            </configuration>
          </execution>
        </executions>
      </plugin>

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-dependency-plugin</artifactId>
        <version>3.1.1</version>
        <executions>
          <execution>
            <id>extract-support-feature</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>unpack</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <artifactItems>
            <artifactItem>
              <groupId>io.openliberty.arquillian</groupId>
              <artifactId>arquillian-liberty-support</artifactId>
              <version>1.0.6</version>
              <type>zip</type>
              <classifier>feature</classifier>
              <overWrite>true</overWrite>
              <outputDirectory>${wlp.usr.dir}</outputDirectory>
            </artifactItem>
          </artifactItems>
        </configuration>
      </plugin>

Expected outcome

Arquillian will start the server via WLPManagedContainer.java

Actual outcome

Line breaks are original.

Mar 25, 2020 4:32:56 PM io.openliberty.arquillian.managed.WLPManagedContainer start
FINER: Starting server with command: [$HOME/svn/project/web/rest-v1/target/liberty/wlp/bin/server, run, ArquillianManagedServer]

Server ArquillianManagedServer d
oes not exist.

Known workarounds

None. At least no easy one.

I tried liberty.env.wlp.usr.dir, but it will only set a server-specific server.env file.
I tried multiple arquillianProperties, but even the java options have no effect because as you can see WLPManagedContainer.java will only read WLP_USER_DIR from either a system-wide server.env or a environment variable.
I can unset the global WLP_USER_DIR by setting the jvm option -Dwlp.ignore.user.dir.from.env=true via arquillianProperties in pom.xml, but this will make liberty look in ${project.build.directory}/liberty/usr again, which is kind of correct, I guess.

Working workaround if, and only if maven-resoures-plugin is defined AFTER the liberty-maven-plugin:

<plugin>
        <artifactId>maven-resources-plugin</artifactId>
        <executions>
          <execution>
            <id>copy-global-server.env</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>copy-resources</goal>
            </goals>
            <configuration>
              <outputDirectory>${project.build.directory}/liberty/wlp/etc</outputDirectory>
              <resources>
                <resource>
                  <directory>${basedir}/src/test/liberty/etc</directory>
                  <filtering>true</filtering>
                </resource>
              </resources>
            </configuration>
          </execution>
        </executions>
      </plugin>

Proposed solution

WLPManagedContainer cannot look up the pom.xml file for the other plugin's userDirectory property. So I think the next best thing would be to have the maven-liberty-plugin write a system-wide server.env file in ${project.build.directory}/liberty/etc/server.env.

I'm confused as to why there appears to be an inconsistency between the value of WLP_USER_DIR environment variable and the property that is set in the pom.xml?

In your issue, your pom references:

<wlp.usr.dir>${project.build.directory}/wlp-usr</wlp.usr.dir>

while your environment variable references:

WLP_USER_DIR=$HOME/.local/share/wlp-usr

When I try with our sample Arquillian guide, it appears to be working if I set:

export WLP_USER_DIR="/path/to/guide-arquillian-managed/finish/target/wlp-usr/"

and then in the pom.xml liberty-maven-plugin configuration:

<userDirectory>${WLP_USER_DIR}</userDirectory.

I have seen the Server ArquillianManagedServer does not exist. message before, although not with that line break. I'm actually not sure where it comes from, but I suspect it could be from the core Arquillian implementation as I can't find that error in our container code. What was strange for me was that it was not finding the server even though it existed in the proper location. It went away after I cleared my .m2 cache and restarted my computer.

The problem seems to be that my export from my .profile seems to override what I set in my pom.xml. In your example you set both to the same value. But in my example, I want to set the pom.xml value relative to the project's target directory. That won't work.

I referenced the corresponding lines in the java file. You can clearly see that the environment variable will override values you set in the pom.xml.

The problem seems to be that my export from my .profile seems to override what I set in my pom.xml.

I'm not sure how that's the case, since I don't see you actually defining or referencing WLP_USER_DIR in the pom.

In your example you set both to the same value. But in my example, I want to set the pom.xml value relative to the project's target directory. That won't work.

My example works as well if you set <wlp.usr.dir>${project.build.directory}/wlp-usr</wlp.usr.dir> and <userDirectory>${wlp.usr.dir}</userDirectory> instead of referencing the system property. The resolved values just have to match.

I referenced the corresponding lines in the java file. You can clearly see that the environment variable will override values you set in the pom.xml.

Not sure what you're referring to here, since we don't even look for the variable value from pom.xml or container configuration so there would be nothing to override? Setting userDirectory in the pom.xml doesn't propagate that variable to the Arquillian container. It is used for more generic runtime configuration outside of Arquillian context.

Regardless, your original observation appears to be correct:

Expected behavior:
Can set WLP_USER_DIR somewow in pom.xml and/or not use WLP_USER_DIR from environment.

So perhaps the solution here is to introduce a new container configuration property that allows you to define the usr directory location. That could be set using arquillianProperties configuration, allowing for centralized configuration from pom.xml that will propagate to Arquillian container.

I'm not sure how that's the case, since I don't see you actually defining or referencing WLP_USER_DIR in the pom.

As I said three times now... It is set by WLPManagedContainer.java.

It is described and linked to the corresponding lines in the first comment. If a env var is defined, it will be used and the setting from the pom.xml is ignored.

Please read the first comment again thoroughly and inspect the linked file from your repository. No offense intended, but I think you did not quite get what I originally meant.

I'm not sure how that's the case, since I don't see you actually defining or referencing WLP_USER_DIR in the pom.

Another try to clarify the issue: I have set WLP_USER_DIR in my $HOME/.profile. Even if you do not set it in your pom.xml, the file WLPManagedContainer.java will read it from your global environment. See my comment:

I tried multiple arquillianProperties, but even the java options have no effect because as you can see WLPManagedContainer.java will only read WLP_USER_DIR from either a system-wide server.env or a environment variable.

Looking at the code, if WLP_USER_DIR is set, the pom.xml is ignored.

   /**
    * Get wlp/usr taking account of user preferences
    *
    * @return wlp.user.dir
    * @throws IOException
    */
   private String getWlpUsrDir() throws IOException {
      String usrDir = getLibertyEnvVar(WLP_USER_DIR);
      if (usrDir == null) {
         usrDir = containerConfiguration.getWlpHome() + "/usr/";
      }
      log.finer("wlp.usr.dir path: " + usrDir);
      return usrDir;
   }

Source: https://github.com/OpenLiberty/liberty-arquillian/blob/master/liberty-managed/src/main/java/io/openliberty/arquillian/managed/WLPManagedContainer.java#L1401-L1414

I think it is not too hard to understand, when looking at the code.

Sorry if there is miscommunication. I consulted with a colleague to talk through your issue and try to understand how to best resolve it.

As you mention in your original post, a working solution involves creating your own server.env file and copying it to the etc folder. You can simplify this configuration by taking advantage of the libertySettingsFolder parameter in the create goal: https://github.com/OpenLiberty/ci.maven/blob/master/docs/create.md

Also to be clear, the userDirectory configuration parameter for the liberty-maven-plugin does not have anything to do with the configuration of the Arquillian container. That is all done though arquillianProperties.

At the moment, there is no way to directly define the usr directory that the Arquillian container should use through the pom.xml.

Hi @chyt - no worries! The best thing I could think of would be to either use the maven-resoure-plugin with filtering to copy a filtered server.env to target/wlp/etc, or to unset WLP_USER_DIR in my mvn shell script.

Both approaches lack portability, and I will not be able to execute tests of other projects - especially when using mvnw instead of mvn.

That said, I think we need a new arquillian property which takes precedence over WLP_USER_HOME, right?

OpenLiberty / liberty-arquillian