search

tag1consulting / goose

Load testing framework, inspired by Locust
https://tag1.com/goose
Apache License 2.0
812 stars 71 forks source link

`--request-log` significantly affects measured throughput #549

Closed sp1ff closed 10 months ago

sp1ff commented 1 year ago

I'm using goose to load-test an app under development & I've noticed something rather surprising: the measured throughput is significantly affected by the presence or absence of the --request-log option. Weirdly, giving the option more than doubles reported QPS.

No idea where to begin trouble-shooting this one.

jeremyandrews commented 1 year ago

I am unable to duplicate this. Is there anything non-standard or otherwise unusual about your configuration/usage? Are you able to duplicate this by enabling any log file, or just --request-log (because they all share the same thread and the same logic). Please provide more detail so that I can better attempt to duplicate.

sp1ff commented 1 year ago

I freely admit, it seems nuts. I can't explain it based on my reading of the code. But the behavior is consistent, and tied to --request-log specifically (I systematically tried all combinations to narrow it down).

Only non-standard thing I can think of is that in one scenario, because I'm testing two requests hitting my endpoint in a particular order, but at the same time, I synthesized one request through GooseUser and another just with reqwest.

Not sure I'd spend too much time on this, since it seems so odd. Maybe tag it as "if seen again, investigate".

jeremyandrews commented 1 year ago

Is this only a Goose reporting anomaly? Or are you actually seeing more load server side? Where exactly are you seeing the difference? Can you provide specific examples of metrics with and without the option enabled?

Requests made directly by reqwest would not be included in the metrics reported by Goose.

jeremyandrews commented 1 year ago

Adding invalid tag to flag that with the information we have nothing more can be done.

jeremyandrews commented 10 months ago

Closing for now -- if you or anyone else runs into this and can answer the above questions, please re-open!