httpteleport

Teleports 10Gbps http traffic over 1Gbps networks. Built on top of fastrpc.

Use cases

httpteleport may significantly reduce inter-server network bandwidth overhead and costs for the following cases:

• RTB servers.
• HTTP-based API servers (aka REST, JSON, JSON-RPC or HTTP-RPC services and microservices).
• Reverse proxies.
• Load balancers.

How does it work?

It just sends batched http requests and responses over a single compressed connection. This solves the following issues:

• High network bandwidth usage
• High network packets rate
• A lot of open TCP connections

Unlike http pipelining, httpteleport responses may be sent out-of-order. This resolves head of line blocking issue.

Links

• Docs

• httptp - standalone single-binary reverse proxy and load balancer based on httpteleport. httptp source code may be used as an example of httpteleport usage.

FAQ

• Q: Why httpteleport doesn't use HTTP/2.0?

  A: Because http/2.0 has many features, which aren't used by httpteleport. More features complicate the code, make it more error-prone and may slow it down.

• Q: Why does httpteleport provide fasthttp- based API instead of standard net/http- based API?

  A: Because httpteleport is optimized for speed. So it have to use fasthttp for http-related stuff to be fast.

• Q: Give me performance numbers.

  A: httpteleport achieves 200K qps on a single CPU core in end-to-end test, where a client sends requests to a local server and the server sends responses back to the client:

$ GOMAXPROCS=1 go test -bench=. -benchmem
goos: linux
goarch: amd64
pkg: github.com/valyala/httpteleport
BenchmarkEndToEndGetNoDelay1              300000          4346 ns/op      60.05 MB/s           0 B/op          0 allocs/op
BenchmarkEndToEndGetNoDelay10             300000          4370 ns/op      59.71 MB/s           3 B/op          0 allocs/op
BenchmarkEndToEndGetNoDelay100            300000          4406 ns/op      59.23 MB/s           6 B/op          0 allocs/op
BenchmarkEndToEndGetNoDelay1000           300000          4457 ns/op      58.55 MB/s          24 B/op          0 allocs/op
BenchmarkEndToEndGetNoDelay10K            300000          5868 ns/op      44.48 MB/s         178 B/op          1 allocs/op
BenchmarkEndToEndGetDelay1ms              300000          4771 ns/op      54.70 MB/s          21 B/op          0 allocs/op
BenchmarkEndToEndGetDelay2ms              200000          7943 ns/op      32.86 MB/s          31 B/op          0 allocs/op
BenchmarkEndToEndGetDelay4ms              200000          7741 ns/op      33.71 MB/s          31 B/op          0 allocs/op
BenchmarkEndToEndGetDelay8ms              200000         10580 ns/op      24.67 MB/s          26 B/op          0 allocs/op
BenchmarkEndToEndGetDelay16ms             100000         16923 ns/op      15.42 MB/s          50 B/op          0 allocs/op
BenchmarkEndToEndGetCompressNone          200000          7899 ns/op      33.04 MB/s          31 B/op          0 allocs/op
BenchmarkEndToEndGetCompressFlate         100000         13257 ns/op      19.69 MB/s         129 B/op          0 allocs/op
BenchmarkEndToEndGetCompressSnappy        200000          8158 ns/op      31.99 MB/s          40 B/op          0 allocs/op
BenchmarkEndToEndGetTLSCompressNone       200000          8692 ns/op      30.02 MB/s          39 B/op          0 allocs/op
BenchmarkEndToEndGetTLSCompressFlate      100000         13710 ns/op      19.04 MB/s         131 B/op          0 allocs/op
BenchmarkEndToEndGetTLSCompressSnappy     200000          8480 ns/op      30.78 MB/s          42 B/op          0 allocs/op
BenchmarkEndToEndGetPipeline1             300000          4673 ns/op      55.85 MB/s           0 B/op          0 allocs/op
BenchmarkEndToEndGetPipeline10            300000          4610 ns/op      56.61 MB/s           3 B/op          0 allocs/op
BenchmarkEndToEndGetPipeline100           300000          4576 ns/op      57.03 MB/s           6 B/op          0 allocs/op
BenchmarkEndToEndGetPipeline1000          300000          4886 ns/op      53.41 MB/s          26 B/op          0 allocs/op