JoeDog / siege

Siege is an http load tester and benchmarking utility
GNU General Public License v3.0
5.9k stars 386 forks source link

-j doesn't always output JSON #213

Open SJrX opened 1 year ago

SJrX commented 1 year ago

This might be user error but in my opinion if you support outputting things in json you should be guaranteed that there is JSON output. In many cases having a log file or if it creates the /root/.siege file it will output to standard out, which means that it's much harder to do anything with the json output.

In a program I have written (that always outputs JSON), anything that is not the JSON response goes to stderr, not standard out. Another option would be to create a new option that lets me specify a file name that will write the JSON report to.

JoeDog commented 1 year ago

Could you provide the conditions under which it doesn't output JSON?

On Mon, Oct 3, 2022 at 9:19 PM Steve Ramage @.***> wrote:

This might be user error but in my opinion if you support outputting things in json you should be guaranteed that there is JSON output. In many cases having a log file or if it creates the /root/.siege file it will output to standard out, which means that it's much harder to do anything with the json output.

In a program I have written (that always outputs JSON), anything that is not the JSON response goes to stderr, not standard out. Another option would be to create a new option that lets me specify a file name that will write the JSON report to.

— Reply to this email directly, view it on GitHub https://github.com/JoeDog/siege/issues/213, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJRHZWEPJRN7GG3OSD4T7TWBOA3FANCNFSM6AAAAAAQ4DI2F4 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Jeff Fulmer 1-717-799-8226 https://www.joedog.org/ He codes

SJrX commented 1 year ago

With this Dockerfile

FROM ubuntu:22.04

RUN apt-get update && apt-get install -y siege jq

Build it and start a new shell

docker build -t siege-test  .
docker run -it --rm siege-test /bin/bash

Then run the following commands.

# siege -c 1 -r 1 -j www.google.ca > report.json
# cat report.json | jq
parse error: Invalid numeric literal at line 1, column 4
# cat report.json
New configuration template added to /root/.siege
Run siege -C to view the current settings in that file
{       "transactions":                            2,
        "availability":                       100.00,
        "elapsed_time":                         0.26,
        "data_transferred":                     0.02,
        "response_time":                        0.13,
        "transaction_rate":                     7.69,
        "throughput":                           0.08,
        "concurrency":                          1.00,
        "successful_transactions":                 2,
        "failed_transactions":                     0,
        "longest_transaction":                  0.21,
        "shortest_transaction":                 0.05
}

If you run it again, it will pass.

Another example is.

#siege -c 1 -r 1 --log=./siege_log.txt -j www.google.ca  > file.json
cat file.txt | jq
{
  "transactions": 2,
  "availability": 100,
  "elapsed_time": 0.18,
  "data_transferred": 0.02,
  "response_time": 0.09,
  "transaction_rate": 11.11,
  "throughput": 0.12,
  "concurrency": 1,
  "successful_transactions": 2,
  "failed_transactions": 0,
  "longest_transaction": 0.13,
  "shortest_transaction": 0.05
}
parse error: Invalid numeric literal at line 14, column 4

#cat file.json
{       "transactions":                            2,
        "availability":                       100.00,
        "elapsed_time":                         0.18,
        "data_transferred":                     0.02,
        "response_time":                        0.09,
        "transaction_rate":                    11.11,
        "throughput":                           0.12,
        "concurrency":                          1.00,
        "successful_transactions":                 2,
        "failed_transactions":                     0,
        "longest_transaction":                  0.13,
        "shortest_transaction":                 0.05
}
LOG FILE: ./siege_log.txt
You can disable this log file notification by editing
/root/.siege/siege.conf and changing 'show-logfile' to false.

In these cases the benefits of having a json formatted output is lost in my opinion, because other output means it can't really be handled programmatically easily.

JoeDog commented 1 year ago

Is this part of the file you're catting?

New configuration template added to /root/.siege Run siege -C to view the current settings in that file

That's from siege.config - There's no need to run that command unless you want create a new config file.

This can be corrected by doing exactly what it says:

LOG FILE: ./siege_log.txt You can disable this log file notification by editing /root/.siege/siege.conf and changing 'show-logfile' to false.

In your config file: show-logfile = false

I will note that as a bug. The LOG FILE: /path message should go to stderr instead of stdout

On Tue, Oct 4, 2022 at 9:31 AM Steve Ramage @.***> wrote:

With this Dockerfile

FROM ubuntu:22.04

RUN apt-get update && apt-get install -y siege jq

Build it and start a new shell

docker build -t siege-test . docker run -it --rm siege-test /bin/bash

Then run the following commands.

siege -c 1 -r 1 -j www.google.ca > report.json

cat report.json | jq

parse error: Invalid numeric literal at line 1, column 4

cat report.json

New configuration template added to /root/.siege Run siege -C to view the current settings in that file { "transactions": 2, "availability": 100.00, "elapsed_time": 0.26, "data_transferred": 0.02, "response_time": 0.13, "transaction_rate": 7.69, "throughput": 0.08, "concurrency": 1.00, "successful_transactions": 2, "failed_transactions": 0, "longest_transaction": 0.21, "shortest_transaction": 0.05 }

If you run it again, it will pass.

Another example is.

siege -c 1 -r 1 --log=./siege_log.txt -j www.google.ca > file.json

cat file.txt | jq { "transactions": 2, "availability": 100, "elapsed_time": 0.18, "data_transferred": 0.02, "response_time": 0.09, "transaction_rate": 11.11, "throughput": 0.12, "concurrency": 1, "successful_transactions": 2, "failed_transactions": 0, "longest_transaction": 0.13, "shortest_transaction": 0.05 } parse error: Invalid numeric literal at line 14, column 4

cat file.json

{ "transactions": 2, "availability": 100.00, "elapsed_time": 0.18, "data_transferred": 0.02, "response_time": 0.09, "transaction_rate": 11.11, "throughput": 0.12, "concurrency": 1.00, "successful_transactions": 2, "failed_transactions": 0, "longest_transaction": 0.13, "shortest_transaction": 0.05 } LOG FILE: ./siege_log.txt You can disable this log file notification by editing /root/.siege/siege.conf and changing 'show-logfile' to false.

In these cases the benefits of having a json formatted output is lost in my opinion, because other output means it can't really be handled programmatically easily.

— Reply to this email directly, view it on GitHub https://github.com/JoeDog/siege/issues/213#issuecomment-1267004896, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJRHZXCZ3EV3OI5C2NJA5DWBQWUXANCNFSM6AAAAAAQ4DI2F4 . You are receiving this because you commented.Message ID: @.***>

-- Jeff Fulmer 1-717-799-8226 https://www.joedog.org/ He codes

JoeDog commented 1 year ago

I pushed out a fix for this

LOG FILE: ./siege_log.txt You can disable this log file notification by editing /root/.siege/siege.conf and changing 'show-logfile' to false.

That will go to stderr now. There's not much I can do to help you with running siege.config. It's a shell script. You can edit it.

On Tue, Oct 4, 2022 at 10:33 AM Jeff Fulmer @.***> wrote:

Is this part of the file you're catting?

New configuration template added to /root/.siege Run siege -C to view the current settings in that file

That's from siege.config - There's no need to run that command unless you want create a new config file.

This can be corrected by doing exactly what it says:

LOG FILE: ./siege_log.txt You can disable this log file notification by editing /root/.siege/siege.conf and changing 'show-logfile' to false.

In your config file: show-logfile = false

I will note that as a bug. The LOG FILE: /path message should go to stderr instead of stdout

On Tue, Oct 4, 2022 at 9:31 AM Steve Ramage @.***> wrote:

With this Dockerfile

FROM ubuntu:22.04

RUN apt-get update && apt-get install -y siege jq

Build it and start a new shell

docker build -t siege-test . docker run -it --rm siege-test /bin/bash

Then run the following commands.

siege -c 1 -r 1 -j www.google.ca > report.json

cat report.json | jq

parse error: Invalid numeric literal at line 1, column 4

cat report.json

New configuration template added to /root/.siege Run siege -C to view the current settings in that file { "transactions": 2, "availability": 100.00, "elapsed_time": 0.26, "data_transferred": 0.02, "response_time": 0.13, "transaction_rate": 7.69, "throughput": 0.08, "concurrency": 1.00, "successful_transactions": 2, "failed_transactions": 0, "longest_transaction": 0.21, "shortest_transaction": 0.05 }

If you run it again, it will pass.

Another example is.

siege -c 1 -r 1 --log=./siege_log.txt -j www.google.ca > file.json

cat file.txt | jq { "transactions": 2, "availability": 100, "elapsed_time": 0.18, "data_transferred": 0.02, "response_time": 0.09, "transaction_rate": 11.11, "throughput": 0.12, "concurrency": 1, "successful_transactions": 2, "failed_transactions": 0, "longest_transaction": 0.13, "shortest_transaction": 0.05 } parse error: Invalid numeric literal at line 14, column 4

cat file.json

{ "transactions": 2, "availability": 100.00, "elapsed_time": 0.18, "data_transferred": 0.02, "response_time": 0.09, "transaction_rate": 11.11, "throughput": 0.12, "concurrency": 1.00, "successful_transactions": 2, "failed_transactions": 0, "longest_transaction": 0.13, "shortest_transaction": 0.05 } LOG FILE: ./siege_log.txt You can disable this log file notification by editing /root/.siege/siege.conf and changing 'show-logfile' to false.

In these cases the benefits of having a json formatted output is lost in my opinion, because other output means it can't really be handled programmatically easily.

— Reply to this email directly, view it on GitHub https://github.com/JoeDog/siege/issues/213#issuecomment-1267004896, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJRHZXCZ3EV3OI5C2NJA5DWBQWUXANCNFSM6AAAAAAQ4DI2F4 . You are receiving this because you commented.Message ID: @.***>

-- Jeff Fulmer 1-717-799-8226 https://www.joedog.org/ He codes

-- Jeff Fulmer 1-717-799-8226 https://www.joedog.org/ He codes

SJrX commented 1 year ago

Thank you.

That will go to stderr now. There's not much I can do to help you with running siege.config. It's a shell script. You can edit it.

This part I didn't understand.

I think maybe the safest thing would be to have a way of just putting the JSON file output somewhere specific. I think that if you have a flag that says output will be json, all output should be json. I'm kind of leary about putting on logging or verbosity with the -j flag. I'm not sure what can handle JSON with non JSON.

But I found a way around this now :) Thanks again.