ConstantThroughputTimer in shared mode all threads start with an incorrect delay of 0

[asfimport]

baerrach (Bug 51480): In shared mode:

• calcMode.4=all active threads (shared)
• calcMode.5=all active threads in current thread group (shared) always has a previousTime of 0 for all threads.

This causes the first run of each thread to not use the "shared" times.

I've provided a patch, with unit tests.

A summary of the changes are below

ConstantThroughputTimer

• Changed threadGroupsInfoMap from Map to ConcurrentHashMap (it was using a ConcurrentHashMap implementation but exposing it as a Map) and putIfAbsent() is needed
• increased serialVersionUID as internal structure has changed.
• Moved ThroughputInfo to its own class and gave it a rewrite
• add constants for calc modes.
• increased MILLISEC_PER_MIN to protected visibility
• replaced previousTime with an instance of ThroughputInfo (this unifies the Timer for all use cases)
• added documentation where it was missing
• delay() delegates to throughputInfo
• deleted calculateCurrentTarget() - the comment indicated it was only used in test code and it no longer is
• renamed calculateDelay to calculateDelayForMode
• moved implementationd details of calculateSharedDelay to ThroughputInfo
• reset no longer resets class variables this is done in testEnded
• reset calls setCalcMode() to ensure that sharing of ThroughputInfo instances are done correctly for the shared modes

ThroughputInfo

• uses a static class SystemClock (which delegates to System.currentTimeMillis) to allow for unit testing
• previousTime from ConstantThroughputTimer renamed to lastScheduledTime
• lastScheduledTime is guarded by "this" instead of a Mutex.
• ConstantThroughputTimer.calculateSharedDelay and ConstantThroughputTimer.delay did the same things but with a different implementation, this is unified into calculateDelay()

== Testing ==

• MockJMeterContext created that delegates all calls to an instance of JMeterContext, used to switch out the current context when "activated" as unit testing doesn't setup threads needed for ThreadLocal context variables.
• Created a simulation class to make creating scenarios from a file possible
• Created ConstantThroughputTimerTest with some more thorough testing scenarios
• Created ConstantThroughputTimerTestData which holds all the test data needed to hide the complexity of setting up unit tests
• Upgraded PackageTest to use SimulationClock instead of relying on Thread.sleep and timing to test
• SimulationClock to control SystemClock timings during test
• SimulationEvent defines what offset the event occurs at and the expected delay
• simulator_mode*.txt files for each mode to run a test scenario for.

Votes in Bugzilla: 2 Severity: normal OS: All

[asfimport]

baerrach (migrated from Bugzilla): Created attachment jmeter-v2_4_bug_51480.patch: Fix with unit tests

[asfimport]

@milamberspace (migrated from Bugzilla): Thanks for your works.

But, your patch doesn't works very well with the svn trunk. There have some errors (when apply patch) with

• ConstantThroughputTimer.java
• PackageTest.java

I have modified some things to get all patch elements in my eclipse source code.

But now, when I try to run ant tests, I have some errors: [java] There were 3 failures: [java] 1) testTimer1(org.apache.jmeter.timers.PackageTest)junit.framework.AssertionFailedError: Expected delay of approx 500 expected:<500.0> but was:<2000.0> [java] at org.apache.jmeter.timers.PackageTest.testTimer1(PackageTest.java:50) [java] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [java] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [java] at org.apache.jorphan.test.AllTests.main(AllTests.java:225) [java] 2) testTimer2(org.apache.jmeter.timers.PackageTest)junit.framework.AssertionFailedError: expected:<1> but was:<1001> [java] at org.apache.jmeter.timers.PackageTest.testTimer2(PackageTest.java:61) [java] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [java] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [java] at org.apache.jorphan.test.AllTests.main(AllTests.java:225) [java] 3) testTimer3(org.apache.jmeter.timers.PackageTest)junit.framework.AssertionFailedError: expected:<10> but was:<19> [java] at org.apache.jmeter.timers.PackageTest.testTimer3(PackageTest.java:74) [java] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [java] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [java] at org.apache.jorphan.test.AllTests.main(AllTests.java:225) [java] FAILURES!!!

Please could you send a new patch (tested it before on a fresh svn trunk & ant tests). Thanks.

[asfimport]

baerrach (migrated from Bugzilla): It looks like the patch doesn't apply cleanly on org.apache.jmeter.timers.ConstantThroughputTimer and org.apache.jmeter.timers.PackageTest

Dont know why.

resupplying patch and these two files individually.

I also needed to update the build.xml to copy across the simulation files.

[asfimport]

baerrach (migrated from Bugzilla): Created attachment jmeter-trunk_bug_51480.patch: Trunk Patch

[asfimport]

baerrach (migrated from Bugzilla): Created attachment ConstantThroughputTimer.java: ConstantThroughputTimer.java

ConstantThroughputTimer.java

````java /* * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements. See the NOTICE file distributed with * this work for additional information regarding copyright ownership. * The ASF licenses this file to You under the Apache License, Version 2.0 * (the "License"); you may not use this file except in compliance with * the License. You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. * */ package org.apache.jmeter.timers; import java.util.concurrent.ConcurrentHashMap; import org.apache.jmeter.engine.event.LoopIterationEvent; import org.apache.jmeter.testbeans.TestBean; import org.apache.jmeter.testelement.AbstractTestElement; import org.apache.jmeter.testelement.TestListener; import org.apache.jmeter.threads.JMeterContextService; import org.apache.jmeter.threads.AbstractThreadGroup; import org.apache.jmeter.util.JMeterUtils; import org.apache.jorphan.logging.LoggingManager; import org.apache.log.Logger; /** * This class implements a constant throughput timer. A Constant Throughtput * Timer paces the samplers under its influence so that the total number of * samples per unit of time approaches a given constant as much as possible. * * There are two different ways of pacing the requests: *

*
• delay each thread according to when it last ran
*
• delay each thread according to when any thread last ran
*

*/ public class ConstantThroughputTimer extends AbstractTestElement implements Timer, TestListener, TestBean { private static final long serialVersionUID = 4; private static final int CALC_MODE_1_THIS_THREAD_ONLY = 0; private static final int CALC_MODE_2_ALL_ACTIVE_THREADS = 1; private static final int CALC_MODE_3_ALL_ACTIVE_THREADS_IN_CURRENT_THREAD_GROUP = 2; private static final int CALC_MODE_4_ALL_ACTIVE_THREADS_SHARED = 3; private static final int CALC_MODE_5_ALL_ACTIVE_THREADS_IN_CURRENT_THREAD_GROUP_SHARED = 4; private static final Logger log = LoggingManager.getLoggerForClass(); protected static final double MILLISEC_PER_MIN = 60000.0; /** * Target time for the start of the next request. The delay provided by the * timer will be calculated so that the next request happens at this time. */ private ThroughputInfo throughputInfo = new ThroughputInfo(); private String calcMode; // String representing the mode // (Locale-specific) private int modeInt; // mode as an integer /** * Desired throughput, in samples per minute. */ private double throughput; //For calculating throughput across all threads private final static ThroughputInfo allThreadsInfo = new ThroughputInfo(); //For holding the ThrougputInfo objects for all ThreadGroups. Keyed by AbstractThreadGroup objects private final static ConcurrentHashMap threadGroupsInfoMap = new ConcurrentHashMap(); /** * Constructor for a non-configured ConstantThroughputTimer. */ public ConstantThroughputTimer() { } /** * Sets the desired throughput. * * @param throughput * Desired sampling rate, in samples per minute. */ public void setThroughput(double throughput) { this.throughput = throughput; } /** * Gets the configured desired throughput. * * @return the rate at which samples should occur, in samples per minute. */ public double getThroughput() { return throughput; } public String getCalcMode() { return calcMode; } // Needed by test code int getCalcModeInt() { return modeInt; } /** * Setting this has the side effect of sharing throughputInfo if the mode is * shared. * * @param mode * the delay calculation mode */ public void setCalcMode(String mode) { this.calcMode = mode; // TODO find better way to get modeInt this.modeInt = ConstantThroughputTimerBeanInfo.getCalcModeAsInt(calcMode); switch (modeInt) { case CALC_MODE_4_ALL_ACTIVE_THREADS_SHARED: throughputInfo = allThreadsInfo; break; case CALC_MODE_5_ALL_ACTIVE_THREADS_IN_CURRENT_THREAD_GROUP_SHARED: final org.apache.jmeter.threads.AbstractThreadGroup group = JMeterContextService .getContext().getThreadGroup(); /* * Share the first thread's throughputInfo */ threadGroupsInfoMap.putIfAbsent(group, throughputInfo); throughputInfo = threadGroupsInfoMap.get(group); break; } } /** * Retrieve the delay to use during test execution. * * @see org.apache.jmeter.timers.Timer#delay() */ public long delay() { return throughputInfo.calculateDelay(calculateDelayForMode()); } /** *

* Calculate the delay based on the mode *

* * @return the delay (how long before another request should be made) in * milliseconds */ private long calculateDelayForMode() { long delay = 0; // N.B. we fetch the throughput each time, as it may vary during a test double msPerRequest = (MILLISEC_PER_MIN / getThroughput()); switch (modeInt) { case CALC_MODE_2_ALL_ACTIVE_THREADS: /* * Each request is allowed to run every msPerRequest. Each thread * can run a request, so each thread needs to be delayed by the * total pool size of threads to keep the expected throughput. */ delay = (long) (JMeterContextService.getNumberOfThreads() * msPerRequest); break; case CALC_MODE_3_ALL_ACTIVE_THREADS_IN_CURRENT_THREAD_GROUP: /* * Each request is allowed to run every msPerRequest. Each thread * can run a request, so each thread needs to be delayed by the * total pool size of threads to keep the expected throughput. */ delay = (long) (JMeterContextService.getContext().getThreadGroup().getNumberOfThreads() * msPerRequest); break; /* * The following modes all fall through for the default */ case CALC_MODE_1_THIS_THREAD_ONLY: case CALC_MODE_4_ALL_ACTIVE_THREADS_SHARED: case CALC_MODE_5_ALL_ACTIVE_THREADS_IN_CURRENT_THREAD_GROUP_SHARED: default: delay = (long) msPerRequest; break; } return delay; } private synchronized void reset() { throughputInfo = new ThroughputInfo(); setCalcMode(calcMode); } /** * Provide a description of this timer class. * * TODO: Is this ever used? I can't remember where. Remove if it isn't -- * TODO: or obtain text from bean's displayName or shortDescription. * * @return the description of this timer class. */ @Override public String toString() { return JMeterUtils.getResString("constant_throughput_timer_memo"); //$NON-NLS-1$ } /** * Get the timer ready to compute delays for a new test. *

* {@inheritDoc} */ public void testStarted() { log.debug("Test started - reset throughput calculation."); reset(); } /** * {@inheritDoc} */ public void testEnded() { /* * There is no way to clean up static variables. The best place to do * that is in the testEnded() call as this wont affect a running test. * If these are in testStarted then they affect already initialised * objects. */ allThreadsInfo.reset(); threadGroupsInfoMap.clear(); } /** * {@inheritDoc} */ public void testStarted(String host) { testStarted(); } /** * {@inheritDoc} */ public void testEnded(String host) { } /** * {@inheritDoc} */ public void testIterationStart(LoopIterationEvent event) { } }````

[asfimport]

baerrach (migrated from Bugzilla): Created attachment PackageTest.java: PackageTest.java

PackageTest.java

````java /* * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements. See the NOTICE file distributed with * this work for additional information regarding copyright ownership. * The ASF licenses this file to You under the Apache License, Version 2.0 * (the "License"); you may not use this file except in compliance with * the License. You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. * */ package org.apache.jmeter.timers; import java.util.ResourceBundle; import org.apache.jmeter.junit.JMeterTestCase; import org.apache.jmeter.testbeans.BeanInfoSupport; import org.apache.jmeter.threads.JMeterContextService; import org.apache.jmeter.threads.TestJMeterContextService; import org.apache.jmeter.util.BeanShellInterpreter; import org.apache.jorphan.logging.LoggingManager; import org.apache.log.Logger; public class PackageTest extends JMeterTestCase { private static final Logger log = LoggingManager.getLoggerForClass(); private SimulationClock simulationClock = new SimulationClock(); public PackageTest(String arg0) { super(arg0); } @Override protected void setUp() throws Exception { super.setUp(); ThroughputInfo.setSystemClock(simulationClock); } @Override protected void tearDown() throws Exception { ThroughputInfo.setSystemClock(new ThroughputInfo.SystemClock()); super.tearDown(); } public void testTimer1() throws Exception { ConstantThroughputTimer timer = new ConstantThroughputTimer(); assertEquals(0,timer.getCalcModeInt());// Assume this thread only timer.setThroughput(60.0);// 1 per second long delay = timer.delay(); // Initialise assertEquals(0,delay); // First one runs immediately simulationClock.increaseTime(500); delay = timer.delay(); log.info("testTimer1: new delay = " + delay); long diff=Math.abs(delay-500); assertTrue("Delay is approximately 500",diff<=50); } public void testTimer2() throws Exception { ConstantThroughputTimer timer = new ConstantThroughputTimer(); assertEquals(0,timer.getCalcModeInt());// Assume this thread only timer.setThroughput(60.0);// 1 per second assertEquals(0, timer.delay()); // First one runs immediately assertEquals(1000,timer.delay()); // Should delay for 1 second simulationClock.increaseTime(1000); timer.setThroughput(60000.0);// 1 per milli-second assertEquals(1,timer.delay()); // Should delay for 1 milli-second } public void testTimer3() throws Exception { ConstantThroughputTimer timer = new ConstantThroughputTimer(); ConstantThroughputTimerBeanInfo bi = new ConstantThroughputTimerBeanInfo(); ResourceBundle rb = (ResourceBundle) bi.getBeanDescriptor().getValue(BeanInfoSupport.RESOURCE_BUNDLE); timer.setCalcMode(rb.getString("calcMode.2")); //$NON-NLS-1$ - all threads assertEquals(1,timer.getCalcModeInt());// All threads assertEquals(0,JMeterContextService.getNumberOfThreads()); for(int i=1; i<=10; i++){ TestJMeterContextService.incrNumberOfThreads(); } assertEquals(10,JMeterContextService.getNumberOfThreads()); timer.setThroughput(600.0);// 10 per second assertEquals(0, timer.delay()); // First one runs immediately assertEquals(1000,timer.delay()); // Should delay for 1 second simulationClock.increaseTime(1000); timer.setThroughput(600000.0);// 10 per milli-second assertEquals(1,timer.delay()); // Should delay for 1 milli-second simulationClock.increaseTime(1); for(int i=1; i<=990; i++){ TestJMeterContextService.incrNumberOfThreads(); } assertEquals(1000,JMeterContextService.getNumberOfThreads()); timer.setThroughput(60000000.0);// 1000 per milli-second assertEquals(1,timer.delay()); // Should delay for 1 milli-second } public void testTimerBSH() throws Exception { if (!BeanShellInterpreter.isInterpreterPresent()){ final String msg = "BeanShell jar not present, test ignored"; log.warn(msg); return; } BeanShellTimer timer = new BeanShellTimer(); long delay; timer.setScript("\"60\""); delay = timer.delay(); assertEquals(60,delay); timer.setScript("60"); delay = timer.delay(); assertEquals(60,delay); timer.setScript("5*3*4"); delay = timer.delay(); assertEquals(60,delay); } } ````

[asfimport]

@milamberspace (migrated from Bugzilla): Thanks for new submission. The new patch file is good (including ConstantThroughputTimer and PackageTest).

ant tests works fine now.

I've create a simple test script, mode 4 works fine, but mode 5 don't start.

Message is : Uncaught Exception java.lang.Error: java.lang.reflect.InvocationTargetException. See log file for details.

Log file : 2011/07/17 10:50:01 ERROR - jmeter.testbeans.TestBeanHelper: This should never happen. java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.jmeter.testbeans.TestBeanHelper.invokeOrBailOut(TestBeanHelper.java:157) at org.apache.jmeter.testbeans.TestBeanHelper.prepare(TestBeanHelper.java:90) at org.apache.jmeter.engine.StandardJMeterEngine.notifyTestListenersOfStart(StandardJMeterEngine.java:205) at org.apache.jmeter.engine.StandardJMeterEngine.run(StandardJMeterEngine.java:341) at java.lang.Thread.run(Thread.java:636) Caused by: java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:924) at org.apache.jmeter.timers.ConstantThroughputTimer.setCalcMode(ConstantThroughputTimer.java:138) ... 9 more

Uncaught Exception java.lang.Error: java.lang.reflect.InvocationTargetException. See log file for details. 2011/07/17 10:50:01 ERROR - jmeter.JMeter: Uncaught exception: java.lang.Error: java.lang.reflect.InvocationTargetException at org.apache.jmeter.testbeans.TestBeanHelper.invokeOrBailOut(TestBeanHelper.java:166) at org.apache.jmeter.testbeans.TestBeanHelper.prepare(TestBeanHelper.java:90) at org.apache.jmeter.engine.StandardJMeterEngine.notifyTestListenersOfStart(StandardJMeterEngine.java:205) at org.apache.jmeter.engine.StandardJMeterEngine.run(StandardJMeterEngine.java:341) at java.lang.Thread.run(Thread.java:636)

Please, for new submission, detab your source code. See: http://wiki.apache.org/jakarta-jmeter/JMeterEclipse

[asfimport]

@milamberspace (migrated from Bugzilla): TG3 don't works.

Created attachment bug_51480_ConstantThroughputTimer.jmx: Simple test script

[asfimport]

baerrach (migrated from Bugzilla): Looking into it.

As an aside, where is the documentation on how to run JMeter? The ant task run_gui fails becaose of missing property files ${basedir}/bin/

• jmeter.properties
• saveservice.properties
• upgrade.properties

I've had to copy these in and update the run_gui target to include <classpath> ... <path refid="run.classpath"/> where <path id="run.classpath"> <fileset dir="${dest.jar}" includes="*.jar"/> </path>

[asfimport]

baerrach (migrated from Bugzilla): Looks like ConstantThroughputTimer:133 case CALC_MODE_5_ALL_ACTIVE_THREADS_IN_CURRENT_THREAD_GROUP_SHARED: final org.apache.jmeter.threads.AbstractThreadGroup group = JMeterContextService .getContext().getThreadGroup();

is returning null for group 2011/07/18 14:00:37 INFO - jmeter.timers.ConstantThroughputTimer: group = null

I didn't notice this, because my real JMeter tests are all CALC_MODE_4_ALL_ACTIVE_THREADS_SHARED

What's the correct way to check is we are in running mode?

[asfimport]

baerrach (migrated from Bugzilla): Using JMeterContextService.getContext().isSamplingStarted

StandardJMeterEngine.run() seems to set this value when starting.

[asfimport]

baerrach (migrated from Bugzilla): ConstantThroughputTimer.setCalcMode becomes:

/**
 * Setting this has the side effect of sharing <code>throughputInfo</code> if the mode is
 * <em>shared</em> and sampling has started.
 *
 * @param mode
 *            the delay calculation mode
 */
public void setCalcMode(String mode) {
    this.calcMode = mode;
    // TODO find better way to get modeInt
    this.modeInt = ConstantThroughputTimerBeanInfo.getCalcModeAsInt(calcMode);

    if (JMeterContextService.getContext().isSamplingStarted()) {
        switch (modeInt) {
        case CALC_MODE_4_ALL_ACTIVE_THREADS_SHARED:
            throughputInfo = allThreadsInfo;
            break;

        case CALC_MODE_5_ALL_ACTIVE_THREADS_IN_CURRENT_THREAD_GROUP_SHARED:
            final org.apache.jmeter.threads.AbstractThreadGroup group = JMeterContextService
                    .getContext().getThreadGroup();
            /*
             * Share the first thread's throughputInfo
             */
            threadGroupsInfoMap.putIfAbsent(group, throughputInfo);
            throughputInfo = threadGroupsInfoMap.get(group);
            break;
        }
    }
}

But my ant tests are no longer working...

[asfimport]

baerrach (migrated from Bugzilla): Sorry, work has been busy. Milamber, where did we get to with this?

[asfimport]

@ham1 (migrated from Bugzilla): Is this still an issue?