yasukata / tinyhttpd-lwip-dpdk

A tiny HTTP server built on lwIP and DPDK
Apache License 2.0
33 stars 7 forks source link

lwip client ? #3

Closed adribr53 closed 1 year ago

adribr53 commented 1 year ago

Hi,

First of all I am grateful to the author for releasing this nice project ! As for my issue :

I considered this project previously for comparison with the linux tcp/ip stack. For doing so, I would need a client in addition to the current server (I would want both hosts communicating to use lwip).

I tried in the past but could not get it done, altough I assumed it would have been an easy job.

I consider giving it another shot, but first I would be curious to know if you think this would be simple, or would it require to modify the code dpdk related (I did not use it yet).

Thank you for your time.

yasukata commented 1 year ago

Thank you for your message.

I have experimentally added the client mode; please find it at commit id 272ebe4099744207b1d59231578bc963c0981b05.

The command for the server mode is the same as before, and the command for the client mode additionally specifies the server IP address and the number of concurrent connections by -s and -c, respectively.

The following is the example output.

server

$ sudo LD_LIBRARY_PATH=./dpdk/install/lib/x86_64-linux-gnu ./app -l 0-1 --proc-type=primary --file-prefix=pmd1 --allow=0000:04:00.1 -- -a 10.0.0.2 -g 10.0.0.1 -m 255.255.255.0 -l 1 -p 10000
EAL: Detected CPU lcores: 16
EAL: Detected NUMA nodes: 2
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/pmd1/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: 1024 hugepages of size 2097152 reserved, but no mounted hugetlbfs found for that size
EAL: VFIO support initialized
EAL: Using IOMMU type 1 (Type 1)
EAL: Ignore mapping IO port bar(2)
EAL: Probe PCI driver: net_ixgbe (8086:10fb) device: 0000:04:00.1 (socket 0)
TELEMETRY: No legacy callbacks, legacy socket not created
http data length: 63 bytes
-- application has started --
[server]:          0 Requests/sec  (rx           0 bps, tx           0 bbs)
[server]:      98434 Requests/sec  (rx    33073824 bps, tx    49610736 bbs)
[server]:     319868 Requests/sec  (rx   107475648 bps, tx   161213472 bbs)
[server]:     318025 Requests/sec  (rx   106856400 bps, tx   160284600 bbs)
[server]:     747972 Requests/sec  (rx   251318592 bps, tx   376977888 bbs)
[server]:    1082936 Requests/sec  (rx   363866496 bps, tx   545799744 bbs)
[server]:    1074529 Requests/sec  (rx   361041744 bps, tx   541562616 bbs)
[server]:    1082924 Requests/sec  (rx   363862464 bps, tx   545793696 bbs)
[server]:    1082777 Requests/sec  (rx   363813072 bps, tx   545719608 bbs)
[server]:    1083066 Requests/sec  (rx   363910176 bps, tx   545865264 bbs)
[server]:    1044888 Requests/sec  (rx   351082368 bps, tx   526623552 bbs)
[server]:    1082642 Requests/sec  (rx   363767712 bps, tx   545651568 bbs)
[server]:    1050398 Requests/sec  (rx   352933728 bps, tx   529400592 bbs)
[server]:    1082571 Requests/sec  (rx   363743856 bps, tx   545615784 bbs)
[server]:    1084397 Requests/sec  (rx   364357392 bps, tx   546536088 bbs)

client

$ sudo LD_LIBRARY_PATH=./dpdk/install/lib/x86_64-linux-gnu ./app -l 0-1 --proc-type=primary --file-prefix=pmd1 --allow=0000:04:00.1 -- -a 10.0.0.3 -g 10.0.0.1 -m 255.255.255.0 -p 10000 -s 10.0.0.2 -c 16
EAL: Detected CPU lcores: 16
EAL: Detected NUMA nodes: 2
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/pmd1/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: 1024 hugepages of size 2097152 reserved, but no mounted hugetlbfs found for that size
EAL: VFIO support initialized
EAL: Using IOMMU type 1 (Type 1)
EAL: Ignore mapping IO port bar(2)
EAL: Probe PCI driver: net_ixgbe (8086:10fb) device: 0000:04:00.1 (socket 0)
TELEMETRY: No legacy callbacks, legacy socket not created
16 concurrent connection(s)
-- application has started --
[client]:          0 Requests/sec  (rx           0 bps, tx           0 bbs)
[client]:     319137 Requests/sec  (rx   160845048 bps, tx   107231040 bbs)
[client]:     319353 Requests/sec  (rx   160953912 bps, tx   107302608 bbs)
[client]:     509913 Requests/sec  (rx   256996152 bps, tx   171335136 bbs)
[client]:    1083112 Requests/sec  (rx   545888448 bps, tx   363925632 bbs)
[client]:    1074288 Requests/sec  (rx   541441152 bps, tx   360960768 bbs)
[client]:    1082550 Requests/sec  (rx   545605200 bps, tx   363736800 bbs)
[client]:    1082810 Requests/sec  (rx   545736240 bps, tx   363824160 bbs)
[client]:    1083246 Requests/sec  (rx   545955984 bps, tx   363970656 bbs)
[client]:    1044359 Requests/sec  (rx   526356936 bps, tx   350904624 bbs)
[client]:    1082979 Requests/sec  (rx   545821416 bps, tx   363880944 bbs)
[client]:    1057230 Requests/sec  (rx   532843920 bps, tx   355229280 bbs)
[client]:    1076742 Requests/sec  (rx   542677968 bps, tx   361785312 bbs)
[client]:    1083462 Requests/sec  (rx   546064848 bps, tx   364043232 bbs)

Regarding the complexity, changes for DPDK-relevant parts are not necessary, I guess.

I think the complicated part of the client implementation is to maintain the state of each HTTP response.

Let's say, the HTTP header of an HTTP response has the following block, which indicates the entire HTTP content is 3000 bytes, and we only received 1000 bytes for the moment.

HTTP/1.1 200 OK
...
Content-Length: 3000
...

In this case, we have to treat the subsequent incoming 2000 bytes (3000 - 1000) as part of the HTTP content, rather than an HTTP header.

To do this, I needed to parse the HTTP header, and maintain the state to know how many bytes of the HTTP content will be sent from the server; this part of the implementation is found in tcp_recv_handler().

I think, if we stop following the HTTP request/response format, we can remove this complexity, and it would be better for simple experimentation on the TCP stack performance.

Anyway, I hope this works in your environment.

adribr53 commented 1 year ago

Thank you for this quick contribution (and sorry for my late answer). I made it run on my setup and might consider using it in some project of mine, so thanks!

yasukata commented 1 year ago

Sounds good! Then, let me close this issue.