search

pawelgaczynski / gain

Gain is a high-performance io_uring networking framework written entirely in Go.
Apache License 2.0
503 stars 22 forks source link

need redis protocol example like below #5

Open kolinfluence opened 1 year ago

kolinfluence commented 1 year ago

need redis protocol example like below https://github.com/IceFireDB/redhub

possible to replace redhub internal networking stuff with gain?

i can sponsor a few coffees for this. p.s. : i can sponsor more coffees if u can do pub/sub features on it.

i was looking at your project for a long time and it's really amazing for u to have done so far. thank you

pawelgaczynski commented 1 year ago

Hi,

It looks like there shouldn't be much trouble replacing gnet with Gain. I'll try to do it for you in my spare time

pawelgaczynski commented 1 year ago

Here you go, an example of using the Gain framework as a backend for the RedHub library:

https://github.com/pawelgaczynski/redhub

Not a perfect implementation, but it should be enough to understand how to use Gain.

kolinfluence commented 1 year ago

cool! will test it out soon and revert with feedback.

kolinfluence commented 1 year ago

redgain is a bit slower. why? ran the example folder code

redis-benchmark -p 6380 -t get,set -n 100000 -q

redhub: SET: 54288.82 requests per second, p50=0.431 msec
GET: 56022.41 requests per second, p50=0.423 msec

redgain: SET: 50505.05 requests per second, p50=0.455 msec
GET: 53475.94 requests per second, p50=0.431 msec

kolinfluence commented 1 year ago

@pawelgaczynski the version of server in example folder is eating around 10% of cpu core doing nothing. why?

pawelgaczynski commented 1 year ago

@kolinfluence please provide the specifications of the host on which you tested it (CPU, Linux, kernel).

kolinfluence commented 1 year ago

@pawelgaczynski

  1. i5-4300U DISTRIB_ID=Ubuntu DISTRIB_RELEASE=22.04 DISTRIB_CODENAME=jammy DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS"

  2. Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz DISTRIB_ID=Ubuntu DISTRIB_RELEASE=22.04 DISTRIB_CODENAME=jammy DISTRIB_DESCRIPTION="Ubuntu 22.04.2 LTS"

pawelgaczynski commented 1 year ago

Please provide the kernel versions of both machines. You can check it with uname -r command. Did you test redgain by always running the client and server on the same machine?

kolinfluence commented 1 year ago

@pawelgaczynski sorry for the missed reply

i5-4300U Linux 5.15.0-73-generic #80-Ubuntu SMP Mon May 15 15:18:26 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

i5-2400 Linux 5.15.0-69-generic #76-Ubuntu SMP Fri Mar 17 17:19:29 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

  1. possible to advise what kernel version to use?
  2. isnt it more accurate to use a separate client / server to test? i'm not very sure but will try again.
kolinfluence commented 1 year ago

@pawelgaczynski can u pls clean up your version of redhub repo so i can try again? thx

# go mod tidy
go: finding module for package github.com/pawelgaczynski/gain
go: finding module for package github.com/rs/zerolog
go: finding module for package github.com/pawelgaczynski/redhub
go: finding module for package github.com/pawelgaczynski/redhub/pkg/resp
go: found github.com/pawelgaczynski/gain in github.com/pawelgaczynski/gain v0.3.1-alpha
go: found github.com/pawelgaczynski/redhub in github.com/pawelgaczynski/redhub v0.0.0-20230525152746-f48c208a8ab1
go: found github.com/pawelgaczynski/redhub/pkg/resp in github.com/pawelgaczynski/redhub v0.0.0-20230525152746-f48c208a8ab1
go: found github.com/rs/zerolog in github.com/rs/zerolog v1.29.1
go: redgain imports
    github.com/pawelgaczynski/redhub: github.com/pawelgaczynski/redhub@v0.0.0-20230525152746-f48c208a8ab1: parsing go.mod:
    module declares its path as: github.com/IceFireDB/redhub
            but was required as: github.com/pawelgaczynski/redhub
root@ubuntu-22:/home/ubuntu/Documents/Programs/redgain/example# grep -rnH 'IceFireDB' .
root@ubuntu-22:/home/ubuntu/Documents/Programs/redgain/example#
kolinfluence commented 1 year ago

can you please modify this also? https://github.com/tidwall/redcon as it has more features

pawelgaczynski commented 1 year ago
  1. possible to advise what kernel version to use?

As io_uring is in active development it is worth using or at least testing performance on the latest possible kernel version. If my spare time permits, I intend to compare gain performance on all official kernel versions from 5.15 upwards.

  1. isnt it more accurate to use a separate client / server to test?

Yes, it is. Client and server on one machine is hardly a common use case for web applications. If you test on separate machines then please indicate which one was the server.

@pawelgaczynski can u pls clean up your version of redhub repo so i can try again? thx

Done.

can you please modify this also?

I'll have a look at it and let you know. I can't promise if and when I'll be able to do it for you, but I'll try.

kolinfluence commented 1 year ago

@pawelgaczynski

Intel(R) Core(TM) i3-8145U CPU @ 2.10GHz DISTRIB_ID=Ubuntu DISTRIB_RELEASE=22.04 DISTRIB_CODENAME=jammy DISTRIB_DESCRIPTION="Ubuntu 22.04.2 LTS"

Linux ubuntu-22 5.19.0-43-generic #44~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon May 22 13:39:36 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

GOMAXPROCS=1 ./server redhub: SET: 149499.17 requests per second GET: 150353.33 requests per second

redgain: SET: 138869.59 requests per second GET: 138159.72 requests per second

using single core performance test. redhub is faster. running redisbenchmark and server on same laptop. it also eats 16-20% cpu being idle, doing nothing

hiqsociety commented 1 year ago

@pawelgaczynski you dont have to benchmark against all kernel version it has been done here, which obviously says the higher kernel version, the faster io_uring performs https://github.com/uNetworking/uWebSockets/issues/1603