net.http.Server or vweb.Server response error when using oha for benchmark

[lost22git]

Describe the bug

A super simple vweb.Server example

• main.v

  
  module main

import vweb

fn main() { vweb.run(&App{}, 8888) }

struct App { vweb.Context }

['/info'; get] fn (mut app App) info() vweb.Result { return app.ok('hi') }

## Build and run app

```shell
╰─❯ v -gc boehm -prod . && .\v_web_server_example.exe
[Vweb] Running app on http://localhost:8888/
[Vweb] We have 16 workers

curl test whether api is ok? (result is ok)

╰─❯ curl  http://localhost:8888/info
hi

benchmark

tool: oha

result:

╰─❯ oha http://localhost:8888/info
Summary:
  Success rate: 0.0000
  Total:        8.0800 secs
  Slowest:      -inf secs
  Fastest:      inf secs
  Average:      0.0000 secs
  Requests/sec: 24.7524

  Total data:   0 B
  Size/request: NaN
  Size/sec:     0 B

Response time histogram:

Latency distribution:
  10% in NaN secs
  25% in NaN secs
  50% in NaN secs
  75% in NaN secs
  90% in NaN secs
  95% in NaN secs
  99% in NaN secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0000 secs, inf secs, -inf secs
  DNS-lookup:   0.0000 secs, inf secs, -inf secs

Status code distribution:

Error distribution:
  [200] No connection could be made because the target machine actively refused it. (os error 10061)

when benchmark net.http.Server also have the same problem

module main

import net
import net.http

fn main() {
    mut server := http.Server{
        port: 8888
        handler: ExampleHandler{}
    }
    server.listen_and_serve()
}

struct ExampleHandler {}

fn (h ExampleHandler) handle(req http.Request) http.Response {
    return http.new_response(http.ResponseConfig{
        header: http.new_header_from_map({http.CommonHeader.content_type:'text/plain'})
        body: 'hi'
    })
}

At the same time, I also found that the memory usage of vweb seems to be quite large （compare to net.http.Server only use 1.8mb）

This is the memory usage after the app starts (benchmark has not been done yet)

• net.http

• vweb

Expected Behavior

always response 200 when benchmark

want to know why vweb.Server use so more memory

Current Behavior

response error：No connection could be made because the target machine actively refused it. (os error 10061)

Reproduction Steps

as above

Possible Solution

No response

Additional Information/Context

No response

V version

V 0.3.4 046dd54

Environment details (OS name and version, etc.)

╰─❯ v doctor
V full version: V 0.3.4 046dd54
OS: windows, Microsoft Windows 11 Pro v22621 64-bit
Processor: 16 cpus, 64bit, little endian,

getwd: E:\code\me\test.v\v_http_server_example
vexe: E:\scoop\global\apps\v\0.3.4\v.exe
vexe mtime: 2023-06-04 18:47:06

vroot: OK, value: E:\scoop\global\apps\v\0.3.4
VMODULES: OK, value: C:\Users\zzz\.vmodules
VTMP: OK, value: C:\Users\zzz\AppData\Local\Temp\v_0

Git version: git version 2.40.1.windows.1
Git vroot status: Error: fatal: not a git repository (or any of the parent directories): .git
.git/config present: false

CC version: Error: exec failed (CreateProcess) with code 2: The system cannot find the file specified.
 cmd: cc --version
thirdparty/tcc: N/A

[lost22git]

Supplement: There is no problem when using hey for the benchmark

[lost22git]

Answer from chatGPT 🤣

There are a few reasons why a Vweb server might use more memory compared to a net/http server:

1. Vweb is built on top of the libuv library, which provides an event-driven programming model with asynchronous I/O. This allows for high performance and scalability, but can result in increased memory usage compared to a traditional blocking I/O approach used by net/http.

2. Vweb includes many features out-of-the-box, such as support for WebSockets, HTTP/2, and automatic TLS encryption. These additional features require additional code and resources, which can increase the memory footprint.

3. Vweb also has a number of optimizations and caching mechanisms that provide improved performance at the expense of increased memory usage. For example, it caches compiled templates and HTML fragments to reduce the amount of work required for subsequent requests.

It's worth noting that while Vweb may use more memory than net/http, this tradeoff is often worthwhile for high-performance web applications that need to handle large volumes of traffic or complex processing requirements.

[lost22git]

Answer from chatGPT 🤣

net.http.Server or vweb.Server response error when using oha for benchmark

Both net/http.Server and Vweb.Server should be capable of handling requests from the oha benchmark tool without error. However, if you are experiencing errors when using oha with either server implementation, there could be a few possible causes:

1. The oha benchmark tool may be misconfigured, or using values that are too high for your server to handle. Make sure that you have set appropriate values for the number of connections, concurrency level, and request rate.

2. Your server may not be properly configured to handle a high volume of requests. You may need to adjust settings such as worker processes, timeouts, or buffer sizes to optimize performance.

3. There may be bugs or other issues in your server implementation that are causing errors under high load. In this case, you should carefully review your code and identify any potential bottlenecks or areas for optimization.

To troubleshoot further, you may want to try running your server implementation with different benchmarks tools, or with lower levels of request traffic to see if the errors persist. Additionally, you can use profiling tools like pprof to identify performance bottlenecks in your code.

[Casper64]

Quick question, does your benchmark return all HTTP 200 responses when you set the number of workers to 1?

vweb.run_at(&App, nr_workers:1, port: 8080)

[lost22git]

@Casper64 same problem

[JalonSolov]

What was the oha command line used to test?

[JalonSolov]

I tried with what is likely the simplest command line, and it appears to be working as expected on Linux. I suspect a Windows specific issue...

$ oha http://localhost:8888/info
Summary:
  Success rate: 1.0000
  Total:        0.0789 secs
  Slowest:      0.0744 secs
  Fastest:      0.0007 secs
  Average:      0.0037 secs
  Requests/sec: 2534.2714

  Total data:   400 B
  Size/request: 2 B
  Size/sec:     4.95 KiB

Response time histogram:
  0.001 [1]   |
  0.008 [188] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.015 [2]   |
  0.023 [0]   |
  0.030 [4]   |
  0.038 [1]   |
  0.045 [2]   |
  0.052 [0]   |
  0.060 [1]   |
  0.067 [0]   |
  0.074 [1]   |

Latency distribution:
  10% in 0.0010 secs
  25% in 0.0013 secs
  50% in 0.0019 secs
  75% in 0.0026 secs
  90% in 0.0035 secs
  95% in 0.0135 secs
  99% in 0.0584 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0001 secs, 0.0000 secs, 0.0005 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0001 secs

Status code distribution:
  [200] 200 responses
$ 

[hatoo]

I've tested it on my Windows 11 and I've found that the V program only accepts connections for ipv6 but not ipv4.

oha -n 1 http://127.0.0.1:8888/info # This fails
oha -n 1 http://localhost:8888/info # Also fails

oha -n 1 http://[::1]:8888/info # OK

[Casper64]

@hatoo the default setting of vweb is to set the AF_INET6 socket option, which allows ipv6 connections. However on most systems the localhost domain translates to the ipv6 adress ::1, so it should be accessible from http://localhost unless this has been changed on your system somehow (or maybe this is because of oha).

You can manually specify to only accept ipv4 or both (AF_UNSPEC)

// ipv4, so it is accessible at 127.0.0.1
vweb.run_at(&App{}, family: .ip)
// unspecified 
vweb.run_at(&App{}, family: .unspec)

[hatoo]

@Casper64 My understanding is slightly different.

It seems that 127.0.0.1 is the default choice for localhost in my environment.

❯ curl -v localhost:8888
*   Trying 127.0.0.1:8888...
...

And we can connect the V program by 127.0.0.1 on Linux (not Windows). Because on Linux, when we use AF_INET6, we can connect using both 127.0.0.1 and [::1] because IPV6_V6ONLY is 0 by default.

[Casper64]

@hatoo In your case if you have to connect via 127.0.0.1 you have to specify the family option to be .ip (AF_INET) as stated in the docs :)

vweb.run_at(&App{}, family: .ip)

I will check the net module if there is a windows specific bug when the socket is bound.