JoeDog / siege

Siege is an http load tester and benchmarking utility
GNU General Public License v3.0
5.9k stars 386 forks source link

-f Flag: POST Requests Fail when JSON Strings Contain Special Characters #189

Open tarskiandhutch opened 3 years ago

tarskiandhutch commented 3 years ago

This error seems to be limited to a very specific use case, but it's one that I'd imagine could come up often for folks load testing backend APIs that manipulate text data.

Here are the conditions:

  1. envoke Siege using -f flag
  2. urls.txt file lists many POST requests
  3. POST requests include JSON strings
  4. JSON strings include a dollar sign $ (and maybe other special chars, but this is where I see the error in my case)

For the request that includes a $, the endpoint logs will show that it only received the content of the request up to (and excluding) the $.

For instance, your urls.txt looks like this (using dummy urls and such because my case is proprietary):

https://site.com POST {"user": "jojo", "sentence": "that is a big apple"}
https://site.com POST {"user": "weehaa", "sentence": "that will cost you $10 for every mile"}
https://site.com POST {"user": "person2", "sentence": "time for change"}
...

And your Siege call is like this:

siege --content-type "application/json" -f urls.txt -c14 -d1

In my case, my logs show that the request body for the request in line 2 before the transaction was: {"user": "weehaa", "sentence": "that will cost you. In other words, Siege appears to have stopped parsing at the $.

If you take line 2 from the above and curl the same request, you would do something like:

curl  -v --header "Content-Type: application/json" POST --data '{"user": "weehaa", "sentence": "that will cost you $10 for every mile"}' https://site.com

In my case, this curl produces a perfectly good response. However, the same call in Siege is not workable (at least not when I submit many URLs using the -f flag). Is there a way to solve this on my end, or is this actually a problem with Siege?

Thanks!

JoeDog commented 3 years ago

It's treating it like a variable.

You can try this:

AMT="$10" https://joedog.org POST {"user": "jojo", "sentence": "that is a big apple"} https://joedog.org POST {"user": "weehaa", "sentence": "that will cost you $(AMT) for every mile"} https://joedog.org POST {"user": "person2", "sentence": "time for change"}

On Tue, Mar 23, 2021 at 1:02 PM tarskiandhutch @.***> wrote:

This error seems to be limited to a very specific use case, but it's one that I'd imagine could come up often for folks load testing backend APIs that manipulate text data.

Here are the conditions:

  1. envoke Siege using -f flag
  2. urls.txt file lists many POST requests
  3. POST requests include JSON strings
  4. JSON strings include a dollar sign $ (and maybe other special chars, but this is where I see the error in my case)

For the request that includes a $, the endpoint logs will show that it only received the content of the request up to (and excluding) the $.

For instance, your urls.txt looks like this (using dummy urls and such because my case is proprietary):

https://site.com POST {"user": "jojo", "sentence": "that is a big apple"}https://site.com POST {"user": "weehaa", "sentence": "that will cost you $10 for every mile"}https://site.com POST {"user": "person2", "sentence": "time for change"} ...

And your Siege call is like this:

siege --content-type "application/json" -f urls.txt -c14 -d1

In my case, my logs show that the request body before the transaction was: {"user": "weehaa", "sentence": "that will cost you . In other words, Siege appears to have stopped parsing at the $.

If you take line 2 from the above and curl the same request, you would do something like:

curl -v --header "Content-Type: application/json" POST --data '{"user": "weehaa", "sentence": "that will cost you $10 for every mile"}' https://site.com

In my case, this curl produces a perfectly good response. However, the same call in Siege is not workable (at least not when I submit many URLs using the -f flag). Is there a way to solve this on my end, or is this actually a problem with Siege?

Thanks!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/JoeDog/siege/issues/189, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJRHZWDOK6UWS4UAWIG2ELTFDCRVANCNFSM4ZVSXP4A .

-- Jeff Fulmer 1-717-799-8226 https://www.joedog.org/ He codes

tarskiandhutch commented 3 years ago

Interesting! Is it just $s then? If so, then I'm just going to avoid $s in our test requests.

Thanks for the quick response!

JoeDog commented 3 years ago

Yes. It's just the dollar sign. I need to refine that parser

On Tue, Mar 23, 2021 at 2:20 PM tarskiandhutch @.***> wrote:

Interesting! Is it just $s then? If so, then I'm just going to avoid $s in our test requests.

Thanks for the quick response!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/JoeDog/siege/issues/189#issuecomment-805126391, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJRHZXQFJ2YZP5757WZ4LTTFDLYRANCNFSM4ZVSXP4A .

-- Jeff Fulmer 1-717-799-8226 https://www.joedog.org/ He codes

tarskiandhutch commented 3 years ago

Good to know in the meantime.

Thanks for a great tool!

JoeDog commented 3 years ago

The fix is in the repo. You can escape dollar signs in your URL. https://site.com POST {"user": "weehaa", "sentence": "that will cost you \$10 for every mile"}