valyala / fasthttp

Fast HTTP package for Go. Tuned for high performance. Zero memory allocations in hot paths. Up to 10x faster than net/http
MIT License
21.91k stars 1.76k forks source link

Use Named Fields and Align Structures to Reduce Memory Usage #1814

Closed gaby closed 3 months ago

gaby commented 3 months ago

@erikdubbelboer Not sure your thoughts on this since it changes the order of fields in some of the public structs.

Affected Public Structs:

This is something we ran into in Fiber https://github.com/gofiber/fiber/pull/3079, we had a lot of places in the code base using positional instead of named fields. This can cause major bugs in the future when the underlying struct changes. I wrote an analyzer to detect and fix these issues. After fixing those, I ran betteralign against the whole code base.

Summary of savings:

/fasthttp/fasthttputil/inmemory_listener.go:16:23: 16 bytes saved: struct with 40 pointer bytes could be 24
/fasthttp/fasthttputil/pipeconns.go:44:16: 24 bytes saved: struct with 312 pointer bytes could be 288
/fasthttp/fasthttputil/pipeconns.go:95:15: 24 bytes saved: struct with 128 pointer bytes could be 104
/fasthttp/stackless/writer.go:41:13: 8 bytes saved: struct with 72 pointer bytes could be 64
/fasthttp/client.go:175:13: 144 bytes saved: struct with 288 pointer bytes could be 144
/fasthttp/client.go:684:17: 144 bytes saved: struct with 368 pointer bytes could be 224
/fasthttp/client.go:951:24: 24 bytes saved: struct with 48 pointer bytes could be 24
/fasthttp/client.go:2034:15: 8 bytes saved: struct with 40 pointer bytes could be 32
/fasthttp/client.go:2089:20: 8 bytes saved: struct with 40 pointer bytes could be 32
/fasthttp/client.go:2167:21: 72 bytes saved: struct with 144 pointer bytes could be 72
/fasthttp/client.go:2281:25: 88 bytes saved: struct with 208 pointer bytes could be 120
/fasthttp/client.go:2312:19: 24 bytes saved: struct with 1288 pointer bytes could be 1264
/fasthttp/cookie.go:67:13: 32 bytes saved: struct with 208 pointer bytes could be 176
/fasthttp/fs.go:253:9: 48 bytes saved: struct with 160 pointer bytes could be 112
/fasthttp/fs.go:504:16: 24 bytes saved: struct with 160 pointer bytes could be 136
/fasthttp/fs.go:522:13: 24 bytes saved: struct with 184 pointer bytes could be 160
/fasthttp/fs.go:847:27: 8 bytes saved: struct with 32 pointer bytes could be 24
/fasthttp/header.go:26:21: 32 bytes saved: struct with 304 pointer bytes could be 272
/fasthttp/header.go:61:20: 24 bytes saved: struct with 344 pointer bytes could be 320
/fasthttp/header.go:3233:20: 16 bytes saved: struct with 88 pointer bytes could be 72
/fasthttp/http.go:38:14: 8 bytes saved: struct with 776 pointer bytes could be 768
/fasthttp/http.go:89:15: 24 bytes saved: struct with 416 pointer bytes could be 392
/fasthttp/lbclient.go:27:15: 8 bytes saved: struct with 48 pointer bytes could be 40
/fasthttp/peripconn.go:9:23: 8 bytes saved: struct with 96 pointer bytes could be 88
/fasthttp/peripconn.go:41:16: 8 bytes saved: struct with 32 pointer bytes could be 24
/fasthttp/peripconn.go:48:19: 8 bytes saved: struct with 24 pointer bytes could be 16
/fasthttp/server.go:148:13: 8 bytes saved: struct of size 560 could be 552
/fasthttp/server.go:585:17: 48 bytes saved: struct with 1440 pointer bytes could be 1392
/fasthttp/tcpdialer.go:128:16: 24 bytes saved: struct with 80 pointer bytes could be 56
/fasthttp/tcpdialer.go:374:26: 8 bytes saved: struct with 32 pointer bytes could be 24
/fasthttp/tcpdialer.go:398:19: 24 bytes saved: struct with 56 pointer bytes could be 32
/fasthttp/uri.go:42:10: 8 bytes saved: struct with 280 pointer bytes could be 272
/fasthttp/workerpool.go:17:17: 56 bytes saved: struct with 144 pointer bytes could be 88
/fasthttp/examples/client/client.go:20:13: 8 bytes saved: struct with 16 pointer bytes could be 8
/fasthttp/fasthttpadaptor/adaptor.go:90:28: 8 bytes saved: struct with 40 pointer bytes could be 32
/fasthttp/prefork/prefork.go:44:14: 16 bytes saved: struct with 96 pointer bytes could be 80
/fasthttp/prefork/prefork.go:166:15: 8 bytes saved: struct with 24 pointer bytes could be 16
/fasthttp/client_test.go:1948:22: 8 bytes saved: struct with 40 pointer bytes could be 32
/fasthttp/client_test.go:3213:13: 8 bytes saved: struct with 48 pointer bytes could be 40
/fasthttp/client_test.go:3346:13: 16 bytes saved: struct with 72 pointer bytes could be 56
/fasthttp/client_test.go:3404:13: 8 bytes saved: struct with 32 pointer bytes could be 24
/fasthttp/client_timing_test.go:19:21: 24 bytes saved: struct with 56 pointer bytes could be 32
/fasthttp/server_test.go:2653:20: 8 bytes saved: struct with 16 pointer bytes could be 8
/fasthttp/server_test.go:4351:17: 8 bytes saved: struct with 16 pointer bytes could be 8
/fasthttp/server_timing_test.go:252:19: 40 bytes saved: struct with 64 pointer bytes could be 24

Total savings: 1336 bytes

gaby commented 3 months ago

I'm not sure why the CI golangci-lint is failing, it runs fine locally.

image

gaby commented 3 months ago

@valyala Thoughts on these changes?

byte0o commented 3 months ago

我不确定为什么 CI golangci-lint 失败了,它在本地运行良好。

图像

@gaby ci error message : tcpdialer.go:129: File is not gofumpt-ed (gofumpt) prefork/prefork.go:45: File is not gofumpt-ed (gofumpt)

This is due to the use of https://github.com/mvdan/gofumpt to implement a stricter fmt. Delete the blank lines of tcpdialer.go:129 and prefork/prefork.go:45 should be able to solve the problem.

gaby commented 3 months ago

我不确定为什么 CI golangci-lint 失败了,它在本地运行良好。 图像

@gaby ci error message : tcpdialer.go:129: File is not gofumpt-ed (gofumpt) prefork/prefork.go:45: File is not gofumpt-ed (gofumpt)

This is due to the use of mvdan/gofumpt to implement a stricter fmt. Delete the blank lines of tcpdialer.go:129 and prefork/prefork.go:45 should be able to solve the problem.

Yeah, I had ran it and it wasn't removing them. I removed them manually. Should work now. Thanks

erikdubbelboer commented 3 months ago

Thanks!