xe1gyq commented 3 years ago

Hi Siege community,

I am looking for ideas to understand the results below. In summary:

In the "docker run" mode, the number of transactions and transaction rate are decreased when switching from http to https, which is expected due to the ssl layer.
But in the "docker service" mode, the results are the opposite, the number of transactions and transaction rate are increased when switching from http to https.

Do you have any idea about:

Reason for this increase in the number of transactions?
Some specific setting at the docker compose file we are missing?

Docker Run Mode

NGINX packed as a container and started

$ docker run --name nginx --rm -it --net=host --entrypoint /bin/bash nginx:latest

$ ./siege -c50 -t30S http://127.0.0.1
New configuration template added to /root/.siege
Run siege -C to view the current settings in that file

{       "transactions":                        86006,
        "availability":                       100.00,
        "elapsed_time":                        29.48,
        "data_transferred":                    50.20,
        "response_time":                        0.02,
        "transaction_rate":                  2917.44,
        "throughput":                           1.70,
        "concurrency":                         49.54,
        "successful_transactions":             86006,
        "failed_transactions":                     0,
        "longest_transaction":                  0.15,
        "shortest_transaction":                 0.00
}

$ ./siege -c50 -t30S https://127.0.0.1
New configuration template added to /root/.siege
Run siege -C to view the current settings in that file

{       "transactions":                        10194,
        "availability":                       100.00,
        "elapsed_time":                        29.98,
        "data_transferred":                     5.95,
        "response_time":                        0.14,
        "transaction_rate":                   340.03,
        "throughput":                           0.20,
        "concurrency":                         48.46,
        "successful_transactions":             10195,
        "failed_transactions":                     0,
        "longest_transaction":                  0.87,
        "shortest_transaction":                 0.02
}

Docker Service Mode

version: '3.3'

services:

  nginx:
    image: nginx:latest
    ports:
      - "80:80"
      - "443:443"
    networks:
      - nginx
    deploy:
      replicas: 1
      placement:
        constraints: [node.role == manager]
      labels:
        - nginx

networks:
  nginx:
    external: true

$ docker service create --name nginx --replicas=1 --publish 80:80 --publish 443:443 nginx:latest

$ ./siege -c50 -t30S http://127.0.0.1
New configuration template added to /root/.siege
Run siege -C to view the current settings in that file

{       "transactions":                        15323,
        "availability":                       100.00,
        "elapsed_time":                        29.43,
        "data_transferred":                     8.94,
        "response_time":                        0.09,
        "transaction_rate":                   520.66,
        "throughput":                           0.30,
        "concurrency":                         48.29,
        "successful_transactions":             15323,
        "failed_transactions":                     0,
        "longest_transaction":                  1.04,
        "shortest_transaction":                 0.00
}

$ ./siege -c50 -t30S https://127.0.0.1
New configuration template added to /root/.siege
Run siege -C to view the current settings in that file

{       "transactions":                        42216,
        "availability":                       100.00,
        "elapsed_time":                        29.92,
        "data_transferred":                    24.64,
        "response_time":                        0.03,
        "transaction_rate":                  1410.96,
        "throughput":                           0.82,
        "concurrency":                         48.75,
        "successful_transactions":             42216,
        "failed_transactions":                     0,
        "longest_transaction":                  1.10,
        "shortest_transaction":                 0.00
}

Thanks for your help on this :)

JoeDog commented 3 years ago

I don't have an explanation but I will say that's fascinating. It's not just marginally better with https, it's noticeably better. Does this have something to do with docker caching? Maybe those sockets we think we're closing are still open and available for re-use.

Jeff

On Tue, Dec 8, 2020 at 10:08 PM Abraham Arce notifications@github.com wrote:

Hi siege community,

I am looking for ideas to understand the results below. In summary:

In the "docker run" mode, the number of transactions and transaction rate are decreased when switching from http to https, which is expected due to the ssl layer.

But in the "docker service" mode, the results are the opposite, the number of transactions and transaction rate are increased when switching from http to https.

Do you have any idea about:

Reason for this increase in the number of transactions?

Some specific setting at the docker compose file we are missing?

Docker Run Mode

NGINX packed as a container and started

$ docker run --name nginx --rm -it --net=host --entrypoint /bin/bash nginx:latest

$ ./siege -c50 -t30S http://127.0.0.1 New configuration template added to /root/.siege Run siege -C to view the current settings in that file

{ "transactions": 86006, "availability": 100.00, "elapsed_time": 29.48, "data_transferred": 50.20, "response_time": 0.02, "transaction_rate": 2917.44, "throughput": 1.70, "concurrency": 49.54, "successful_transactions": 86006, "failed_transactions": 0, "longest_transaction": 0.15, "shortest_transaction": 0.00 }

$ ./siege -c50 -t30S https://127.0.0.1 New configuration template added to /root/.siege Run siege -C to view the current settings in that file

{ "transactions": 10194, "availability": 100.00, "elapsed_time": 29.98, "data_transferred": 5.95, "response_time": 0.14, "transaction_rate": 340.03, "throughput": 0.20, "concurrency": 48.46, "successful_transactions": 10195, "failed_transactions": 0, "longest_transaction": 0.87, "shortest_transaction": 0.02 }

Docker Service Mode

version: '3.3' services:

nginx: image: marceline.zpn.intel.com/nginx:latest ports:

"80:80"

"443:443" networks:

nginx deploy: replicas: 1 placement: constraints: [node.role == manager] labels:

nginx networks: nginx: external: true

$ docker service create --name nginx --replicas=1 --publish 80:80 --publish 443:443 nginx:latest

$ ./siege -c50 -t30S http://127.0.0.1 New configuration template added to /root/.siege Run siege -C to view the current settings in that file

{ "transactions": 15323, "availability": 100.00, "elapsed_time": 29.43, "data_transferred": 8.94, "response_time": 0.09, "transaction_rate": 520.66, "throughput": 0.30, "concurrency": 48.29, "successful_transactions": 15323, "failed_transactions": 0, "longest_transaction": 1.04, "shortest_transaction": 0.00 }

$ ./siege -c50 -t30S https://127.0.0.1 New configuration template added to /root/.siege Run siege -C to view the current settings in that file

{ "transactions": 42216, "availability": 100.00, "elapsed_time": 29.92, "data_transferred": 24.64, "response_time": 0.03, "transaction_rate": 1410.96, "throughput": 0.82, "concurrency": 48.75, "successful_transactions": 42216, "failed_transactions": 0, "longest_transaction": 1.10, "shortest_transaction": 0.00 }

Thanks for your help on this :)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/JoeDog/siege/issues/184, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJRHZRNRERTAPLFCJ7BEA3ST3S2JANCNFSM4US3OANA .

xe1gyq commented 3 years ago

Thanks @JoeDog for your answer, it is the next idea to work on so let me spend some time learning about it and I will get back to this thread. Any other suggestion, please, let me know :)

JoeDog commented 3 years ago

What docker networking is being used - bridge, host, overlay, etc.?

On Wed, Dec 9, 2020 at 10:06 AM Abraham Arce notifications@github.com wrote:

Thanks @JoeDog https://github.com/JoeDog for your answer, it is the next idea to work on so let me spend some time learning about it and I will get back to this thread. Any other suggestion, please, let me know :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/JoeDog/siege/issues/184#issuecomment-741830739, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJRHZTFGYX64IN6KA44BETST6G77ANCNFSM4US3OANA .

xe1gyq commented 3 years ago

For Docker Run, using Host driver:

$ docker run --name nginx --rm -it --net=host --entrypoint /bin/bash nginx:latest

Not sure about Docker Service, here is the output:

$ docker service create --name nginx --replicas=1 --publish 80:80 --publish 443:443 nginx:latest
l2hnpbjbzp320jf3e7yfb8u1a
overall progress: 1 out of 1 tasks
1/1: running   [==================================================>]
verify: Service converged
$ docker service ps nginx
ID                  NAME                IMAGE                   NODE                  DESIRED STATE       CURRENT STATE            ERROR               PORTS
40dhwnn3y3cy        nginx.1             nginx:latest            wordpress.novalocal   Running             Running 10 seconds ago
$ docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
d24c4033c339        bridge              bridge              local
3e6a5f1e1adb        docker_gwbridge     bridge              local
432d8a64411f        host                host                local
ya7wgpl5f7n1        ingress             overlay             swarm
c9453b914666        none                null                local

I have added the scenario of Docker Swarm, using Overlay driver, same numbers as in Docker Service:

$ docker stack deploy -c docker-compose.yml nginx
network "nginx" is declared as external, but could not be found. You need to create a swarm-scoped network before the stack is deployed
$ docker network create --driver host nginx
Error response from daemon: only one instance of "host" network is allowed
$ docker network create --driver bridge nginx
9815bb2cf93c951f58a3b959bc8627f336dcdd1e4e6f4336240c1e1c02641c7f
$ docker stack deploy -c docker-compose.yml nginx
network "nginx" is declared as external, but it is not in the right scope: "local" instead of "swarm"
$ docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
d24c4033c339        bridge              bridge              local
3e6a5f1e1adb        docker_gwbridge     bridge              local
432d8a64411f        host                host                local
ya7wgpl5f7n1        ingress             overlay             swarm
9815bb2cf93c        nginx               bridge              local
c9453b914666        none                null                local
$ docker network rm 9815bb2cf93c
9815bb2cf93c

$ docker network create --driver overlay nginx
tgmqme48gbdwasjxjq1d03h17
$ docker stack deploy -c docker-compose.yml nginx
Creating service nginx_nginx
$ docker stack ps nginx
ID                  NAME                IMAGE          NODE                  DESIRED STATE       CURRENT STATE            ERROR               PORTS
35ccj8mmsvh3        nginx_nginx.1       nginx:latest   wordpress.novalocal   Running             Running 21 seconds ago

$ ./siege -c50 -t30S http://127.0.0.1

{       "transactions":                        15305,
        "availability":                       100.00,
        "elapsed_time":                        29.55,
        "data_transferred":                     8.93,
        "response_time":                        0.09,
        "transaction_rate":                   517.94,
        "throughput":                           0.30,
        "concurrency":                         48.24,
        "successful_transactions":             15305,
        "failed_transactions":                     0,
        "longest_transaction":                  1.06,
        "shortest_transaction":                 0.00
}
$ ./siege -c50 -t30S https://127.0.0.1

{       "transactions":                        41754,
        "availability":                       100.00,
        "elapsed_time":                        29.97,
        "data_transferred":                    24.37,
        "response_time":                        0.03,
        "transaction_rate":                  1393.19,
        "throughput":                           0.81,
        "concurrency":                         48.03,
        "successful_transactions":             41754,
        "failed_transactions":                     0,
        "longest_transaction":                  1.11,
        "shortest_transaction":                 0.00
}

xe1gyq commented 3 years ago

@JoeDog you gave me another idea with your questions, it was to test the Official NGINX Docker image and results are also increased when going from http to https in Docker Swarm mode, same as it was initially reported.

Docker Run Mode

$ ./siege -c50 -t30S http://127.0.0.1

{       "transactions":                       559795,
        "availability":                       100.00,
        "elapsed_time":                        29.03,
        "data_transferred":                   326.72,
        "response_time":                        0.00,
        "transaction_rate":                 19283.33,
        "throughput":                          11.25,
        "concurrency":                         48.71,
        "successful_transactions":            559796,
        "failed_transactions":                     0,
        "longest_transaction":                  0.83,
        "shortest_transaction":                 0.00
}

$ ./siege -c50 -t30S https://127.0.0.1

{       "transactions":                        34507,
        "availability":                       100.00,
        "elapsed_time":                        29.86,
        "data_transferred":                    20.14,
        "response_time":                        0.04,
        "transaction_rate":                  1155.63,
        "throughput":                           0.67,
        "concurrency":                         49.90,
        "successful_transactions":             34507,
        "failed_transactions":                     0,
        "longest_transaction":                  0.07,
        "shortest_transaction":                 0.02
}

Docker Service Mode

$ ./siege -c50 -t30S http://127.0.0.1

{       "transactions":                        13587,
        "availability":                       100.00,
        "elapsed_time":                        29.88,
        "data_transferred":                     7.93,
        "response_time":                        0.11,
        "transaction_rate":                   454.72,
        "throughput":                           0.27,
        "concurrency":                         49.01,
        "successful_transactions":             13587,
        "failed_transactions":                     0,
        "longest_transaction":                  1.03,
        "shortest_transaction":                 0.00
}

$ ./siege -c50 -t30S https://127.0.0.1

{       "transactions":                        32038,
        "availability":                       100.00,
        "elapsed_time":                        29.08,
        "data_transferred":                    18.70,
        "response_time":                        0.05,
        "transaction_rate":                  1101.72,
        "throughput":                           0.64,
        "concurrency":                         49.92,
        "successful_transactions":             32038,
        "failed_transactions":                     0,
        "longest_transaction":                  1.09,
        "shortest_transaction":                 0.01
}

JoeDog / siege

Siege Results from Two Docker Modes #184

Docker Run Mode

Docker Service Mode

Docker Run Mode

Docker Service Mode