search

yandex / yandex-tank

Load and performance benchmark tool
Other
2.43k stars 281 forks source link

Using yandex.tank with jmeter: different results in overload service and jmeter log #436

Open rightchoice89 opened 6 years ago

rightchoice89 commented 6 years ago

I'm trying to use yandex.tank with a jmeter as a load generator and overload service for storing test results but I encountered a strange behaviour:

I had launched a simple test and after the test completion I saw that RPS results in linux terminal are the same as in the overload. Then I took a look into a jmeter_*.jtl log that had produced during test execution in /var/loadtest/test_dir - it contained much more requests (with http code 200) then in terminal / overload.

I used different versions of yandex.tank - 1.8.35, 1.9.0, 1.9.1 and jmeter 3.0, with and without using proxy for uploading results to overload, in docker or on the server itself - the result was the same: jmeter_*.jtl log contains much more requests with successfull answers than I can see in overload.

The config I've used is the following:

[tank]
plugin_phantom=
plugin_jmeter=yandextank.plugins.JMeter
plugin_uploader=yandextank.plugins.DataUploader overload
[jmeter]
jmeter_path=/home/user/apache-jmeter-3.0/bin/jmeter.sh
jmx=/home/ansible/test_plan.jmx
[overload]
token_file=/home/user/ovrld_token.txt

Also I tried a "buffered_seconds" option - it didn't help.

Could anyone give me a piece of advice, please?

Best regards.

direvius commented 6 years ago

@f2nd have you ever experienced something like this?

rightchoice89 commented 6 years ago

Hey guys,

I'll be very appreciated if you advise me smth. If you need some example artifacts I can provide them as well as a link to the test results in overload service, for compareness.

May be you used yandex.tank with jmeter by yourself and didn't faced this behaviour - in this case I will be grateful if you tell me the versions of software you had used, I will try them.

f2nd commented 6 years ago

@rightchoice89 could you provide jtl file and tank log for tests with count mismatch problems? I run a lot of tests with jmeter and haven't seen such problem in 1.8.35 or 1.9.1

rightchoice89 commented 6 years ago

Hi, @f2nd Here it is - https://yadi.sk/d/P15Sw0a83N8XHY

rightchoice89 commented 6 years ago

@f2nd Any ideas / recommendations, please?