Threading Issue with JMeter and JBrowserDriver

[michael-logan]

Hi,

I have a JMeter test, running a Java Sampler test, which in turn uses a JBrowserDriver. I'm noticing my results are not correct.

Here is the basic rundown of my JMeter test:

• Grab a Username and Password from "CSV Data Set Config"
• Create JBrowserDriver
• Log into website and verify
• Store the driver in memory
• Return JMeter result

I am noticing that the result returned after using the JBrowserDriver is the last result. Trying to explain this issue is a little difficult since I haven't pinned down the issue. I notice the problem doesn't occur if I don't create the JBrowserDriver object. Removing the step of "Store the driver in memory" didn't fix the issue.

Assume my CSV contains these values: user1,password1 user2,password2 user3,password3

And my Java Sampler returns results with Log in success and username like this: Log in=Success,Username=user1

I am getting these kind of results: Log in=Success,Username=user2 Log in=Success,Username=user2

If I change the Thread Count to 3 I get these values: Log in=Success,Username=user3 Log in=Success,Username=user3 Log in=Success,Username=user3

Here is my "Java Sampler" code. Am I instantiating the driver incorrectly? Am I storing it into JMeter context incorrectly since it's a separate process? If I remove the JBrowserDriver instantiation the issue doesn't appear.

`package gov.mypkg;

import org.apache.jmeter.protocol.java.sampler.AbstractJavaSamplerClient; import java.io.Serializable; import java.util.concurrent.TimeUnit; import com.machinepublishers.jbrowserdriver.JBrowserDriver; import com.machinepublishers.jbrowserdriver.Settings;

import org.apache.commons.lang.BooleanUtils; import org.apache.jmeter.config.Arguments; import org.apache.jmeter.samplers.SampleResult; import org.openqa.selenium.*; import org.openqa.selenium.htmlunit.HtmlUnitDriver; import org.openqa.selenium.support.ui.ExpectedConditions; import org.openqa.selenium.support.ui.WebDriverWait; import org.apache.jmeter.protocol.java.sampler.JavaSamplerContext;

public class InstantiateBrowser extends AbstractJavaSamplerClient implements Serializable {

private static final long serialVersionUID = 5710042664127564753L;
private JBrowserDriver driver;
private String lastStep;
public static String sessionUsername = "";
public static String sessionPassword = "";
public static String Host = "";
public static String Directory = "";
public static boolean StackTraceInResult = false;
public static long MaxWaitTimeSeconds = 0;
public static String TestValue = "";
String executionTag;

@Override
public Arguments getDefaultParameters() {
    // add arguements to the JMeter Java Sample page.
    Arguments defaultParameters = new Arguments();
    defaultParameters.addArgument("theUsername", "username to use");
    defaultParameters.addArgument("thePassword", "username to use");
    defaultParameters.addArgument("Host", "https://mycompany.com");
    defaultParameters.addArgument("Directory", "appdir");
    defaultParameters.addArgument("StackTraceInResult", "false");
    defaultParameters.addArgument("MaxWaitTimeSeconds", "20");
    return defaultParameters;
}

public void setUp(JavaSamplerContext context) throws Exception {
    // set the execution tag for log statements
    executionTag = "ExecutionTimestamp::" + Common.GetTimeStamp();

    // try to get the parameters that were passed in.
    try {
        sessionUsername = context.getParameter("theUsername");
        sessionPassword = context.getParameter("thePassword");
        Host = context.getParameter("Host");
        Directory = context.getParameter("Directory");
        StackTraceInResult = BooleanUtils.toBoolean(context.getParameter("StackTraceInResult"));
        MaxWaitTimeSeconds = context.getLongParameter("MaxWaitTimeSeconds");
    } catch (Exception ex) {
        Common.ExceptionToSysOut(executionTag, ex);
        Common.wl(executionTag, "Exception with parameters");
    }

    // instantiate the browser
    try {
        driver = new
                JBrowserDriver(Settings.builder()
                        .javaOptions("-XX:+PrintCommandLineFlags", "-Xmx250m")
                        .ajaxWait(MaxWaitTimeSeconds*1000)
                        .build());
    } catch (Exception ex) {
        Common.ExceptionToSysOut(executionTag, ex);
    }

}

@Override
public SampleResult runTest(JavaSamplerContext context) {

    // Create the result
    SampleResult result = new SampleResult();
    result.setDataType(org.apache.jmeter.samplers.SampleResult.TEXT);

    // start the timer
    result.sampleStart(); // start stopwatch

    try {

        // run setUp
        setUp(context);

        // access the website
        driver.get(Host + "/" + Directory);

        lastStep = "Trying to log in...";

        driver.findElement(By.id("usernameTextBox")).clear();
        driver.findElement(By.id("usernameTextBox")).sendKeys(sessionUsername);
        driver.findElement(By.name("passwordTextBox")).clear();
        driver.findElement(By.name("passwordTextBox")).sendKeys(sessionPassword);
        driver.findElement(By.xpath("//button[contains(.,'Sign In')]")).click();

        // Validate that we see the session as logged in.
        // Use my version of fluent find to wait for an element to appear.
        FluentFindResult ffresult = FluentFind.FindFluently(executionTag, driver, MaxWaitTimeSeconds,
                By.xpath("//td[contains(.,'Logged In As')]"), StackTraceInResult);

        if (ffresult.FindResult != null) {
            result.setSuccessful(true);
            result.setResponseMessage(TestValue + sessionUsername + ":" + lastStep + ":" + "Verification Passed");

            // grab the JMeter context and store the browser into memory
            org.apache.jmeter.threads.JMeterContext jmetercontext = org.apache.jmeter.threads.JMeterContextService
                    .getContext();
            org.apache.jmeter.threads.JMeterVariables vars = jmetercontext.getVariables();

            // we are storing the driver as a variable named "browser"
            vars.putObject("browser", driver);
            jmetercontext.setVariables(vars);

        } else {
            result.setSuccessful(false);
            result.setResponseCode("500");
            result.setResponseMessage(TestValue + sessionUsername + ":" + ffresult.FindError);
        }

        // close the driver if we were unable to successfully log in.
        if (!result.isSuccessful()) {
            if (!(driver == null)) {
                driver.quit();
            }
        }

        // stop the clock
        result.sampleEnd();

    } catch (Exception e) {

        result.sampleEnd(); // stop stopwatch
        result.setSuccessful(false);
        result.setResponseCode("500");

    }

    // return the result
    return result;
}

@Override
public void teardownTest(JavaSamplerContext context) {
    // driver.quit();
    super.teardownTest(context);
}

} `

[hollingsworthd]

Sounds like JMeter runs multiple threads? JBrowserDriver is multi-process so you'd probably want the number of processes to match the number of JMeter threads. Settings.builder().processes(numThreads)

JBrowserDriver tracks the number of processes by incrementing a counter when the instance is created and decrementing it when the instance quits. If you create an instance that would exceed the max, the constructor blocks.

[hollingsworthd]

Also these separate processes are transparent to users. All the classes you can access such as JBrowserDriver reside in the parent process--then they in turn communicate over RMI to the child process.

[hollingsworthd]

One more thing, this should probably be documented better, but the max and counter of processes is per Settings object. So if you build new Settings in each thread, the max should be 1. If you share the same built Settings across threads then the max should be the number of threads.

Overall, I don't know that any of this fixes your problem. Just a guess.

[michael-logan]

Hi @hollingsworthd

You are correct, JMeter runs multiple Threads. With each Thread (User in JMeter terms), will run my Java Sampler class (the code above). So if I run 3 threads for my Test I have 3 seperate Java processes for the JBrowserDriver.

I think I have nailed down the issue. If the Ajax wait is higher than 999 it happens more often than not. Values around 4000 make it happen all the time.

driver = new JBrowserDriver(Settings.builder().ajaxWait(4000).build());

I tried assigning the ajaxWait with .ajaxWait(4000L) and it didn't help. This doesn't make sense but it is what I'm seeing.

[michael-logan]

Oh, forgot to mention:

I did add the processes(1) to the Settings and I tried the processes(numThreadsFromJMeter). In both instances the issue still occurred.

[hollingsworthd]

The ajax wait means that that much time must pass between any ajax requests before the page is considered loaded (essentially... it's a little more nuanced when considering iframes, redirects, failed requests, etc).

So if you had a client side script that pinged the server every 750ms, then an ajax wait of 1000 would mean the page would never be considered done loading by the driver and it would block forever. And if you had a wait of 30000 then that's high enough there might be unexpected issues with other timeouts at play.

The default I think is 150ms. That's sufficient for many use cases and environments. If a page has no ajax at all, a value of 0ms is fine. And I'm not that familiar with FluentWait but I think it tries to accomplish the same goal (in a very different way) so maybe 0ms makes sense if you're using FluentWait on an ajax page.

[hollingsworthd]

I think you want these non-static:

public static String sessionUsername = "";
public static String sessionPassword = "";
public static String Host = "";
public static String Directory = "";
public static boolean StackTraceInResult = false;
public static long MaxWaitTimeSeconds = 0;
public static String TestValue = "";