search

myzhan / locust4j

Locust4j is a load generator for locust, written in Java.
MIT License
81 stars 30 forks source link

What version(s) of locustio are supported for the masters in distributed mode #9

Closed lolwhitaker closed 5 years ago

lolwhitaker commented 5 years ago

Hello. What versions of locustio are supported for the master when running in distributed mode?

When I run my slaves against 0.12.1 master, and I receive a ReadTimeout on a slave task, I frequently see this error reported from the master and its not recorded properly in the web ui:

[2019-11-05 17:26:19,758] ip-10-51-78-73/INFO/locust.runners: Sending hatch jobs to 5 ready clients
[2019-11-05 17:26:22,312] ip-10-51-78-73/ERROR/stderr: Traceback (most recent call last):
[2019-11-05 17:26:22,313] ip-10-51-78-73/ERROR/stderr: File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run
[2019-11-05 17:26:22,313] ip-10-51-78-73/ERROR/stderr: File "/home/ubuntu/.local/lib/python2.7/site-packages/locust/runners.py", line 387, in client_listener
[2019-11-05 17:26:22,313] ip-10-51-78-73/ERROR/stderr: events.slave_report.fire(client_id=msg.node_id, data=msg.data)
[2019-11-05 17:26:22,313] ip-10-51-78-73/ERROR/stderr: File "/home/ubuntu/.local/lib/python2.7/site-packages/locust/events.py", line 34, in fire
[2019-11-05 17:26:22,313] ip-10-51-78-73/ERROR/stderr: handler(**kwargs)
[2019-11-05 17:26:22,313] ip-10-51-78-73/ERROR/stderr: File "/home/ubuntu/.local/lib/python2.7/site-packages/locust/stats.py", line 600, in on_slave_report
[2019-11-05 17:26:22,314] ip-10-51-78-73/ERROR/stderr: entry = StatsEntry.unserialize(stats_data)
[2019-11-05 17:26:22,314] ip-10-51-78-73/ERROR/stderr: File "/home/ubuntu/.local/lib/python2.7/site-packages/locust/stats.py", line 405, in unserialize
[2019-11-05 17:26:22,314] ip-10-51-78-73/ERROR/stderr: setattr(obj, key, data[key])
[2019-11-05 17:26:22,314] ip-10-51-78-73/ERROR/stderr: KeyError: 'num_none_requests'
[2019-11-05 17:26:22,314] ip-10-51-78-73/ERROR/stderr: 2019-11-05T17:26:22Z
[2019-11-05 17:26:22,314] ip-10-51-78-73/ERROR/stderr: 
[2019-11-05 17:26:22,314] ip-10-51-78-73/ERROR/stderr: <Greenlet at 0x7fdf6579c158: <bound method MasterLocustRunner.client_listener of <locust.runners.MasterLocustRunner object at 0x7fdf6578ded0>>> failed with KeyError
[2019-11-05 17:26:26,368] ip-10-51-78-73/INFO/locust.runners: Slave ip-10-51-78-238_0220ff2b03bbbc1c91800b7b0ba756dc failed to send heartbeat, setting state to missing.
[2019-11-05 17:26:26,368] ip-10-51-78-73/INFO/locust.runners: Slave ip-10-51-78-238_e2aa790224e6a7e6e6a4d6507e16aee5 failed to send heartbeat, setting state to missing.
[2019-11-05 17:26:26,368] ip-10-51-78-73/INFO/locust.runners: Slave ip-10-51-78-238_a92507d37bd5ca89067babee4df7f3fc failed to send heartbeat, setting state to missing.
[2019-11-05 17:26:26,369] ip-10-51-78-73/INFO/locust.runners: Slave ip-10-51-78-238_37c521fec7104ac830e46eb9e82fb350 failed to send heartbeat, setting state to missing.
[2019-11-05 17:26:26,369] ip-10-51-78-73/INFO/locust.runners: Slave ip-10-51-78-238_6f3e1b09ce3ca2428552929d5a2fe351 failed to send heartbeat, setting state to missing.

When I run against a 0.12.2 master, the error looks like this:

[2019-11-05 17:01:43,741] ip-10-51-78-73/INFO/locust.runners: Sending hatch jobs to 5 ready clients
[2019-11-05 17:02:25,716] ip-10-51-78-73/ERROR/stderr: Traceback (most recent call last):
[2019-11-05 17:02:25,717] ip-10-51-78-73/ERROR/stderr: File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run
[2019-11-05 17:02:25,717] ip-10-51-78-73/ERROR/stderr: File "/home/ubuntu/.local/lib/python2.7/site-packages/locust/runners.py", line 360, in client_listener
[2019-11-05 17:02:25,717] ip-10-51-78-73/ERROR/stderr: events.slave_report.fire(client_id=msg.node_id, data=msg.data)
[2019-11-05 17:02:25,717] ip-10-51-78-73/ERROR/stderr: File "/home/ubuntu/.local/lib/python2.7/site-packages/locust/events.py", line 34, in fire
[2019-11-05 17:02:25,717] ip-10-51-78-73/ERROR/stderr: handler(**kwargs)
[2019-11-05 17:02:25,718] ip-10-51-78-73/ERROR/stderr: File "/home/ubuntu/.local/lib/python2.7/site-packages/locust/stats.py", line 578, in on_slave_report
[2019-11-05 17:02:25,718] ip-10-51-78-73/ERROR/stderr: global_stats.errors[error_key] = StatsError.from_dict(error)
[2019-11-05 17:02:25,718] ip-10-51-78-73/ERROR/stderr: File "/home/ubuntu/.local/lib/python2.7/site-packages/locust/stats.py", line 531, in from_dict
[2019-11-05 17:02:25,718] ip-10-51-78-73/ERROR/stderr: data["occurrences"]
[2019-11-05 17:02:25,718] ip-10-51-78-73/ERROR/stderr: KeyError: 'occurrences'
[2019-11-05 17:02:25,718] ip-10-51-78-73/ERROR/stderr: 2019-11-05T17:02:25Z
[2019-11-05 17:02:25,718] ip-10-51-78-73/ERROR/stderr: 
[2019-11-05 17:02:25,718] ip-10-51-78-73/ERROR/stderr: <Greenlet at 0x7ff475682578: <bound method MasterLocustRunner.client_listener of <locust.runners.MasterLocustRunner object at 0x7ff47563cb90>>> failed with KeyError
[2019-11-05 17:02:29,223] ip-10-51-78-73/INFO/locust.runners: Slave ip-10-51-78-238_849f28d9ddd56bba629998f553e34770 failed to send heartbeat, setting state to missing.
[2019-11-05 17:02:29,223] ip-10-51-78-73/INFO/locust.runners: Slave ip-10-51-78-238_15a0898d89d93894938e50e3a10ecece failed to send heartbeat, setting state to missing.
[2019-11-05 17:02:29,223] ip-10-51-78-73/INFO/locust.runners: Slave ip-10-51-78-238_4fe0e0a4cc179528915ce45261fcd873 failed to send heartbeat, setting state to missing.
[2019-11-05 17:02:29,223] ip-10-51-78-73/INFO/locust.runners: Slave ip-10-51-78-238_5cbaa23ff1e82b80317e34339301c773 failed to send heartbeat, setting state to missing.
[2019-11-05 17:02:29,223] ip-10-51-78-73/INFO/locust.runners: Slave ip-10-51-78-238_ab8e8d9579b16324e08941ed9d8ddd6e failed to send heartbeat, setting state to missing.

As far as I can tell, this error is not something I can fix with changes to my Task implementation and suspect it may be due to incompatibility with some newer versions of the locustio master? Any insight appreciated.

I don't see these errors running against an 0.11.0 locustio master.

lolwhitaker commented 5 years ago

I don't see these errors running against an 0.11.0 locustio master.

myzhan commented 5 years ago

Thanks, check this out https://github.com/myzhan/locust4j/commit/536a66da5d6ddb337a71dc2eae42f82afa14497b .

And I'm releasing 1.0.7.

lolwhitaker commented 5 years ago

Thank you! I just tested this with 1.0.7 and 0.12.2 master and I am no longer seeing the exceptions.