search

golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
124.3k stars 17.7k forks source link

x/text/unicode/norm: reduce init-time memory usage #26752

Open bradfitz opened 6 years ago

bradfitz commented 6 years ago

The golang.org/x/text/unicode/norm package allocs 21.47kB in its init function, regardless of whether the package is ever used.

We should do it lazily, as done in e.g. https://go-review.googlesource.com/c/net/+/127275 and https://go-review.googlesource.com/c/go/+/127075

This would help people who explicitly import it (or import it via golang.org/x/net/http2), as then the std-vendored copy would not need to also allocate the 20KB.

Perhaps some of those tables should be const strings, too, so the linker can put it in read-only sections. Then we can slice into it at runtime to get integers out.

/cc @mpvl

ianlancetaylor commented 6 years ago

CC @mpvl

kevinburke commented 6 years ago

Going to take a crack at this!

gopherbot commented 6 years ago

Change https://golang.org/cl/127924 mentions this issue: unicode/norm: lazy load decomps table

gopherbot commented 6 years ago

Change https://golang.org/cl/127926 mentions this issue: make recompMap a string

agnivade commented 5 years ago

Is there a way to generate the tables for all unicode versions ? If I run "go generate", it only generates the tables for the current Go version.

Also, how are you checking the memory allocation of the init function ? If I just do a blank import and run a mem profile, nothing gets caught.

bradfitz commented 5 years ago

@agnivade, see the top comment on #26775 and https://twitter.com/bradfitz/status/1024726819725950976

gopherbot commented 5 years ago

Change https://golang.org/cl/207017 mentions this issue: unicode/norm: make decomps to a const string