Post requests not functioning properly

lanceofwhichwedream commented 5 years ago

Description of issue

Locust v0.11.0 appears to be using GET requests even though locust file specifies to use a post request

Expected behavior

Post request made to specified URI

Actual behavior

Get request is made instead

Environment settings

OS: Debian devirative
Python version: 3.6.8
Locust version: 0.11.0

Steps to reproduce (for bug reports)

#!/usr/bin/env python3
import json

from locust import HttpLocust, TaskSet, task

headers = {
    "Host": "host.host.net",
    "User-Agent":
    "Mozilla/5.0 (X11; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0",
    "Accept": "application/json, text/plain, */*",
    "Accept-Language": "en-US,en;q=0.5",
    "Accept-Encoding": "gzip, deflate, br",
    "Content-Type": "application/json;charset=utf-8",
    "Authorization":
    "REDACTED",                                                        
    "refreshTokenId": "REDACTED",
    "Content-Length": "49",
    "Connection": "keep-alive",
}
data = {"UserID": "REDACTED"}

def info(l):
    response = l.client.post("/api/",
                             data=data,
                             headers=headers)
    print(response)
    print(response.text)
    print(response.status_code)

class UserBehavior(TaskSet):
    tasks = {info: 1}

class MyLocust(HttpLocust):
    task_set = UserBehavior
    min_wait = 5000
    max_wait = 15000

Sample of the logs

[2019-06-18 16:09:07,290] Eris/INFO/locust.runners: All locusts hatched: MyLocust: 5
[2019-06-18 16:09:07,303] Eris/INFO/stdout: <Response [405]>
[2019-06-18 16:09:07,304] Eris/INFO/stdout:
[2019-06-18 16:09:07,304] Eris/INFO/stdout: {"message":"The requested resource does not support http method 'GET'."}                                                                        
[2019-06-18 16:09:07,304] Eris/INFO/stdout:
[2019-06-18 16:09:07,304] Eris/INFO/stdout: 405
[2019-06-18 16:09:07,304] Eris/INFO/stdout:
[2019-06-18 16:09:15,109] Eris/ERROR/stderr: KeyboardInterrupt
[2019-06-18 16:09:15,110] Eris/ERROR/stderr: 2019-06-18T20:09:15Z
[2019-06-18 16:09:15,110] Eris/ERROR/stderr:
[2019-06-18 16:09:15,110] Eris/INFO/locust.main: Shutting down (exit code 0), bye.
[2019-06-18 16:09:15,110] Eris/INFO/locust.main: Cleaning up runner...
[2019-06-18 16:09:15,127] Eris/INFO/locust.main: Running teardowns...

lanceofwhichwedream commented 5 years ago

I've tested on version 0.9.0 and the error still occurs

[2019-06-18 16:35:57,174] Eris/INFO/stdout: <Response [405]>
[2019-06-18 16:35:57,175] Eris/INFO/stdout: 
[2019-06-18 16:35:57,175] Eris/INFO/stdout: {"message":"The requested resource does not support http method 'GET'."}
[2019-06-18 16:35:57,175] Eris/INFO/stdout: 
[2019-06-18 16:35:57,175] Eris/INFO/stdout: 405
[2019-06-18 16:35:57,176] Eris/INFO/stdout: 
[2019-06-18 16:36:03,653] Eris/ERROR/stderr: KeyboardInterrupt
[2019-06-18 16:36:03,653] Eris/ERROR/stderr: 2019-06-18T20:36:03Z
[2019-06-18 16:36:03,653] Eris/ERROR/stderr: 
[2019-06-18 16:36:03,653] Eris/INFO/locust.main: Shutting down (exit code 0), bye.
[2019-06-18 16:36:03,654] Eris/INFO/locust.main: Cleaning up runner...
[2019-06-18 16:36:03,654] Eris/INFO/locust.main: Running teardowns...
➜  ~ locust --version                                                           
[2019-06-18 16:36:54,111] Eris/INFO/stdout: Locust 0.9.0
[2019-06-18 16:36:54,112] Eris/INFO/stdout:

cgoldberg commented 5 years ago

are you hitting a redirect? that would make requests issue a GET. (Locust uses Requests as the default http client) see: https://github.com/kennethreitz/requests/issues/1704

lanceofwhichwedream commented 5 years ago

To my knowledge there isn't a redirect involved in this, however would Requests and (by extension) Locust handle an application behind a loadbalancer similar to a redirect?

ldelossa commented 5 years ago

I'm having this issue too. Threw me in a giant loop until I logged out my server I was testing. Locusio is definitely sending GETs for POSTs. In my case I'm using the json= flag.

heyman commented 5 years ago

@lanceofwhichwedream Yes, Locust should have no trouble with a load balancer. A load balancer (or reverse proxy) should be mostly transparent to the clients and does not issue any HTTP redirects by itself.

@ldelossa Could you post a minimal code example producing this? I'm not able to reproduce this issue with the code in the original post (I ran the exact same code, and only edited the Host header to make a request to a requestbin.com bin: https://requestbin.com/r/enj414fza79v/1SNYGpbh5MMBAbVnCbJdJV6ZUJC).

It does sound like it could be that the server returns a HTTP redirect, which the client by default follows, and thus (correctly) issues a HTTP GET.

richkettle commented 3 years ago

I seem to also be getting this issue. Is there any update?

cyberw commented 3 years ago

I seem to also be getting this issue. Is there any update?

This ticket never made any sense, so it is unlikely that you are having the same issue. Please file a new ticket with your full details (locust file and locust version mainly), or ask on stack overfloow.

ldelossa commented 3 years ago

IIRC, the GET in these errors occur because of a redirect at the server. So you send a POST, but server responds with a redirect and locust io logs a GET. Its not a real issue.

cyberw commented 3 years ago

IIRC, the GET in these errors occur because of a redirect at the server. So you send a POST, but server responds with a redirect and locust io logs a GET. Its not a real issue.

If that is what happens then it kind of is an issue. It should still log a POST (it might not do that, but in that case I think that behaviour should be changed)

ldelossa commented 3 years ago

Just throwing something out there, is locust io telling the "requests" library to follow redirects?

https://gist.github.com/cbare/4333273#file-test_requests_redirect-py-L20

take note on that "strict" mode.

cyberw commented 3 years ago

I’m guessing we dont, but probably should :)

ldelossa commented 3 years ago

could be it!

richkettle commented 3 years ago

So the server was issuing a redirect. It was only at the apache level. It was a http vs https issue.

Confusing things further was CURL. It doesn't change the verb to GET but continues to use POST. Postman on the other hand does what locust does and moves to GET if POST gets redirected.

I've read up on it a little and it the different behaviours among different tech is frustrating and there are good points for both. Hopefully this comment is found and helps someone else.

Im unsure on whether locust should log a GET or a POST. It feels like it should do... something.

Right now it logs a POST even if it ends up performing a GET.

vcolanoKH commented 2 years ago

In case it's not clear from @richkettle's comment, if you are encountering this issue try switching your target url from using http to using https. This worked in my case.

BTripp1986 commented 2 years ago

Maybe this is something that should be documented somewhere in the Locust docs? Luckily I found this issue, but it wasn't until after much failure(I didn't find this issue until I specifically searched "Locust post sending get".

This may be normal behavior from certain clients like requests, but I think a note somewhere visible in the docs would be helpful. Happy to put a docs PR in this week if others think this would be a welcome addition to the docs.

locustio / locust