search

akara / faban

A framework for developing workload drivers and automation of performance tests
http://faban.sunsource.net/
90 stars 39 forks source link

All run.xml should override cycletime/thinktime in benchmark #66

Closed bhushan2901 closed 10 years ago

bhushan2901 commented 10 years ago

I think it was fixed by but somehow I was not able to find it in latest source. Please see https://java.net/jira/browse/FABAN-10

shanti commented 10 years ago

This is really not a good idea. The idea of a benchmark is that it be repeatable. If you change the cycle/think times randomly, you will get completely different results. The best way to do this is by creating a new benchmark with different think/cycle times.

ScottOaks commented 10 years ago

My 0.02 for why I contributed that to the other faban project. First, it's not all about benchmarking; it's about load generation. Being able to script load generators without compiling code is a good thing.

Second, I'd take a look at Faban commercial competitors like LoadRunner or Oracle Application Testing Suite. They all work on the notion of a test, and a scenario for the test. In faban, that essentially maps to a driver (code), and a run.xml file (scenario parameters). Those tools all allow switching of think times and similar parameters in the scenario without changing the essential driver code.

shanti commented 10 years ago

Thanks Scott. Will check in the fix.

On Tue, Apr 15, 2014 at 8:14 PM, Scott Oaks notifications@github.comwrote:

My 0.02 for why I contributed that to the other faban project. First, it's not all about benchmarking; it's about load generation. Being able to script load generators without compiling code is a good thing.

Second, I'd take a look at Faban commercial competitors like LoadRunner or Oracle Application Testing Suite. They all work on the notion of a test, and a scenario for the test. In faban, that essentially maps to a driver (code), and a run.xml file (scenario parameters). Those tools all allow switching of think times and similar parameters in the scenario without changing the essential driver code.

— Reply to this email directly or view it on GitHubhttps://github.com/akara/faban/issues/66#issuecomment-40558980 .

shanti commented 10 years ago

I have fixed this issue taking Scott's work. But I don't have the cycles or environment to test it currently. Can I request someone to try the fix out - it's in branch ISSUE-66 on github.

To test, you need to modify your run.xml to explicitly specify/change the think/cycle times as follows:

FixedTime MyOperation1, MyOperation2 ThinkTIME 100 0

For multiple cycleDistribution,

FixedTime MyOperation1 CycleTime 100 0 NegativeExponential MyOperation2 ThinkTIME 500 0

Do a run, check if there are no errors and whether the new values are being picked up.

Thanks Shanti

On Mon, Apr 21, 2014 at 5:38 PM, Shanti Subramanyam (gmail) < shanti.subramanyam@gmail.com> wrote:

Thanks Scott. Will check in the fix.

On Tue, Apr 15, 2014 at 8:14 PM, Scott Oaks notifications@github.comwrote:

My 0.02 for why I contributed that to the other faban project. First, it's not all about benchmarking; it's about load generation. Being able to script load generators without compiling code is a good thing.

Second, I'd take a look at Faban commercial competitors like LoadRunner or Oracle Application Testing Suite. They all work on the notion of a test, and a scenario for the test. In faban, that essentially maps to a driver (code), and a run.xml file (scenario parameters). Those tools all allow switching of think times and similar parameters in the scenario without changing the essential driver code.

— Reply to this email directly or view it on GitHubhttps://github.com/akara/faban/issues/66#issuecomment-40558980 .

shanti commented 10 years ago

Fixed.