search

the-benchmarker / web-frameworks

Which is the fastest web framework?
MIT License
6.91k stars 641 forks source link

Hard-code concurrency levels #3029

Closed waghanza closed 3 years ago

waghanza commented 3 years ago

Hi @i4004,

Thanks for opening the issue #3028

The issue raised was due to a dynamic concurrency level selection.

As I'm working on a 8 CPU, I have not the problem, but it seems that you have 12 CPU.

There is no need of a dynamic selection of currency levels, so I hard code it.

Let me know if this fix your bug

Closes #3028

i4004 commented 3 years ago
  1. Got an error on shards build about required brackets, changed to: 
    yaml.scalar "../../bin/client --language #{language} --framework #{name} -r GET:/ -r GET:/user/0 -r POST:/user -h `cat ip.txt` -c 64 -c 256 -c 512"
  2. Still error: 
    csharp.simplify.web [0/1] ⠤ running (66%) > ../../bin/client --language csharp --framework simplify.web -r GET:/ -r GET:/user/0 -r POST:/user -h `cat ip.txt` -c 64 -c 256 -c 512
[Error] 'csharp.simplify.web' failed with status code (256)
[Error]  -- STDOUT(.neph/csharp.simplify.web/log/log.out) --
---> 8845550065cf
Step 11/12 : ENV COMPlus_TieredCompilation 1
---> Using cache
---> cce7ae2d8ff0
Step 12/12 : CMD dotnet /usr/src/app/out/web.dll
---> Using cache
---> 29eee1ecb279
Successfully built 29eee1ecb279
Successfully tagged csharp.simplify.web:latest
Start @ 64
[Error]  -- STDERR(.neph/csharp.simplify.web/log/log.err) --
from ../../tools/src/client.cr:86:32 in 'run'
from ../../tools/src/client.cr:22:1 in 'parse_and_run'
from ../../tools/src/client.cr:22:1 in 'run'
from ../../tools/src/client.cr:116:1 in '__crystal_main'
from ../../../../../usr/share/crystal/src/crystal/main.cr:105:5 in 'main_user_code'
from ../../../../../usr/share/crystal/src/crystal/main.cr:91:7 in 'main'
from ../../../../../usr/share/crystal/src/crystal/main.cr:114:3 in 'main'
from __libc_start_main
from _start
from ???

log.err

Unhandled exception: Empty enumerable (Enumerable::EmptyError)
  from ../../../../../usr/share/crystal/src/enumerable.cr:502:13 in 'first'
  from ../../tools/src/client.cr:86:32 in 'run'
  from ../../tools/src/client.cr:22:1 in 'parse_and_run'
  from ../../tools/src/client.cr:22:1 in 'run'
  from ../../tools/src/client.cr:116:1 in '__crystal_main'
  from ../../../../../usr/share/crystal/src/crystal/main.cr:105:5 in 'main_user_code'
  from ../../../../../usr/share/crystal/src/crystal/main.cr:91:7 in 'main'
  from ../../../../../usr/share/crystal/src/crystal/main.cr:114:3 in 'main'
  from __libc_start_main
  from _start
  from ???
waghanza commented 3 years ago

in which version of crystal are you ?

i4004 commented 3 years ago

Crystal 0.35.1 [5999ae29b] (2020-06-19)

waghanza commented 3 years ago

Could you give me your whole command ? The way you build neph.yaml ...

i4004 commented 3 years ago 
shards build
Dependencies are satisfied
Building: db
Building: neph
Building: client
Building: make
Error target make failed to compile:
In tools/src/make.cr:263:33

 263 | yaml.scalar '../../bin/client --language #{language} --framework #{name} -r GET:/ -r GET:/user/0 -r POST:/user -h `cat ip.txt` -c 64 -c 256 -c 512'
                   ^
Error: unterminated char literal, use double quotes for strings
waghanza commented 3 years ago

Yes, I've fixed this. Mix ' and ", my bad

waghanza commented 3 years ago

Empty enumerable occurs when wrk has could not connect to host

could you check the output of docker ps ? and then docker inspect $CID_FOUND_BEFORE ?

i4004 commented 3 years ago 
docker ps
CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS               NAMES
498986c1f07b        csharp.simplify.web   "/bin/sh -c 'dotnet …"   About an hour ago   Up About an hour                        boring_galileo
95c18c8f15c0        csharp.simplify.web   "/bin/sh -c 'dotnet …"   2 hours ago         Up 2 hours                              ecstatic_wright
afa37d521444        csharp.simplify.web   "/bin/sh -c 'dotnet …"   3 hours ago         Up 3 hours                              festive_kalam
3eba8810428b        csharp.simplify.web   "/bin/sh -c 'dotnet …"   3 hours ago         Up 3 hours                              wizardly_fermat
bee12b29b2c1        csharp.simplify.web   "/bin/sh -c 'dotnet …"   3 hours ago         Up 3 hours                              inspiring_goldwasser
b8bcbe40d8ba        csharp.simplify.web   "/bin/sh -c 'dotnet …"   4 hours ago         Up 4 hours                              eager_volhard
docker inspect 498986c1f07b
[
    {
        "Id": "498986c1f07b43edaed11c2c576c19fae8717a79327aebf99b8533f5258e87d2",
        "Created": "2020-06-24T16:24:20.8276266Z",
        "Path": "/bin/sh",
        "Args": [
            "-c",
            "dotnet /usr/src/app/out/web.dll"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 16551,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2020-06-24T16:24:21.0848602Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:29eee1ecb2793df149c906b45841abbe0c18015fc55642da75a7fa49a35ede68",
        "ResolvConfPath": "/var/lib/docker/containers/498986c1f07b43edaed11c2c576c19fae8717a79327aebf99b8533f5258e87d2/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/498986c1f07b43edaed11c2c576c19fae8717a79327aebf99b8533f5258e87d2/hostname",
        "HostsPath": "/var/lib/docker/containers/498986c1f07b43edaed11c2c576c19fae8717a79327aebf99b8533f5258e87d2/hosts",
        "LogPath": "/var/lib/docker/containers/498986c1f07b43edaed11c2c576c19fae8717a79327aebf99b8533f5258e87d2/498986c1f07b43edaed11c2c576c19fae8717a79327aebf99b8533f5258e87d2-json.log",
        "Name": "/boring_galileo",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": null,
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Capabilities": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/7f2537f7a0e2e62371571aeea5bc2d0dcf1e2fb56cee79a643385a0ae071cb8f-init/diff:/var/lib/docker/overlay2/e0258e8fee8b215ddbb578c7add1bb0af764a3fde7717eae8358372f4d6b758b/diff:/var/lib/docker/overlay2/2194e861080a5c8803d9c46b4de5313a1b3bb67c2ebca39bf60cc4cebc751363/diff:/var/lib/docker/overlay2/3717e8a63cedeed2ed699d78ecaf15d54aab5a057314b7fd5f4f98249573c56a/diff:/var/lib/docker/overlay2/7a39bcebf25568294a5debfafd80cb01689574a4df7a43a5b620388c4df48f33/diff:/var/lib/docker/overlay2/dcf2e569ff54f27a38f0496b37c2ba27fcc6a30dcd8177f4c89746cc80cc7e73/diff:/var/lib/docker/overlay2/764658162a656f61fa949adf6d3bbe510dad7e82aabe2593ec26cdd022a64ed2/diff",
                "MergedDir": "/var/lib/docker/overlay2/7f2537f7a0e2e62371571aeea5bc2d0dcf1e2fb56cee79a643385a0ae071cb8f/merged",
                "UpperDir": "/var/lib/docker/overlay2/7f2537f7a0e2e62371571aeea5bc2d0dcf1e2fb56cee79a643385a0ae071cb8f/diff",
                "WorkDir": "/var/lib/docker/overlay2/7f2537f7a0e2e62371571aeea5bc2d0dcf1e2fb56cee79a643385a0ae071cb8f/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [],
        "Config": {
            "Hostname": "498986c1f07b",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": true,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "ASPNETCORE_URLS=http://*:3000",
                "DOTNET_RUNNING_IN_CONTAINER=true",
                "DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=true",
                "COMPlus_TieredCompilation=1"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "dotnet /usr/src/app/out/web.dll"
            ],
            "Image": "csharp.simplify.web",
            "Volumes": null,
            "WorkingDir": "/usr/src/app",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {
                "desktop.docker.io/wsl-distro": "Ubuntu-20.04"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "e0f18d808baf156faa84828e01b1d66e59effbf7ad9ad536c2e0882d01e439a5",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/e0f18d808baf",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "d3d0cef74ebce8b25150fe0756a2e880719f410072235f13bc15b75d794710f1",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.7",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:07",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "1345e1448e020949efd4303f79317adf7517d8c8c27cc770849cc5f99735958d",
                    "EndpointID": "d3d0cef74ebce8b25150fe0756a2e880719f410072235f13bc15b75d794710f1",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.7",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:07",
                    "DriverOpts": null
                }
            }
        }
    }
]
waghanza commented 3 years ago

Could you check that the file ip.txt in csharp/.simplify.web contains the above IP (172.17.0.7) ?

i4004 commented 3 years ago

Yes, it contains that.

waghanza commented 3 years ago

Could you run the whole commands

shards build
bin/make config
export DATABASE_URL=postgresql://postgres@localhost/benchmark
dropdb -U postgres benchmark
createdb -U postgres benchmark 
psql -U postgres -d benchmark < .ci/dump.sql 
bin/neph csharp.simplify.web
bin/db to_readme
i4004 commented 3 years ago

Done, but when I reached bin/neph csharp.simplify.web, I got the same error

-- // --
Neph is running (0.2.12) 0%||||||||||||||||||||||||||||||||||||||||||||||||||
csharp.simplify.web [0/1] ⠴ running (66%) > ../../bin/client --language csharp --framework simplify.web -r GET:/ -r GET:/user/0 -r POST:/user -h `cat ip.txt` -c 64 -c 256 -c 512
[Error] 'csharp.simplify.web' failed with status code (256)
[Error]  -- STDOUT(.neph/csharp.simplify.web/log/log.out) --
 ---> 8845550065cf
Step 11/12 : ENV COMPlus_TieredCompilation 1
 ---> Using cache
 ---> cce7ae2d8ff0
Step 12/12 : CMD dotnet /usr/src/app/out/web.dll
 ---> Using cache
 ---> 29eee1ecb279
Successfully built 29eee1ecb279
Successfully tagged csharp.simplify.web:latest
Start @ 64
[Error]  -- STDERR(.neph/csharp.simplify.web/log/log.err) --
  from ../../tools/src/client.cr:86:32 in 'run'
  from ../../tools/src/client.cr:22:1 in 'parse_and_run'
  from ../../tools/src/client.cr:22:1 in 'run'
  from ../../tools/src/client.cr:116:1 in '__crystal_main'
  from ../../../../../usr/share/crystal/src/crystal/main.cr:105:5 in 'main_user_code'
  from ../../../../../usr/share/crystal/src/crystal/main.cr:91:7 in 'main'
  from ../../../../../usr/share/crystal/src/crystal/main.cr:114:3 in 'main'
  from __libc_start_main
  from _start
  from ???

log.err

Unhandled exception: Empty enumerable (Enumerable::EmptyError)
  from ../../../../../usr/share/crystal/src/enumerable.cr:502:13 in 'first'
  from ../../tools/src/client.cr:86:32 in 'run'
  from ../../tools/src/client.cr:22:1 in 'parse_and_run'
  from ../../tools/src/client.cr:22:1 in 'run'
  from ../../tools/src/client.cr:116:1 in '__crystal_main'
  from ../../../../../usr/share/crystal/src/crystal/main.cr:105:5 in 'main_user_code'
  from ../../../../../usr/share/crystal/src/crystal/main.cr:91:7 in 'main'
  from ../../../../../usr/share/crystal/src/crystal/main.cr:114:3 in 'main'
  from __libc_start_main
  from _start
  from ???
waghanza commented 3 years ago

could wrk reach the host ? wrk 172.17.0.7:3000

i4004 commented 3 years ago

hmm

wrk 172.17.0.7:3000
invalid URL: 172.17.0.7:3000

I use wrk 4.1.0-4-g0896020 compiled from sources (https://github.com/wg/wrk/wiki/Installing-wrk-on-Linux#ubuntudebian-clean-box) maybe i need to play with wrk to get problem fixed.

i4004 commented 3 years ago 
wrk http://172.17.0.7:3000
unable to connect to 172.17.0.7:3000 Connection timed out
waghanza commented 3 years ago

I do not know why you can not reach 172.17.0.7 as the container is created ... probably something around docker-desktop, which is not supported

could you docker-machine => https://github.com/the-benchmarker/web-frameworks/issues/3028#issuecomment-649411018

i4004 commented 3 years ago

Ty, will try.

waghanza commented 3 years ago

Ok. I merge and close. Feel free to open an other issue if you have any further problems