Open bingoohuang opened 5 years ago
go继承与覆盖
type HelloWorld interface {
Hello()
World()
}
type NullHelloWorld struct{}
func (*NullHelloWorld) Hello() { fmt.Println("Null Hello") }
func (*NullHelloWorld) World() { fmt.Println("Null World") }
type myHello struct {
NullHelloWorld
}
func (*myHello) Hello() { fmt.Println("my Hello") }
func TestMyHello(t *testing.T) {
var mh HelloWorld = new(myHello)
mh.Hello() // my Hello
mh.World() // Null World
}
当涉及到内存管理时,Go已经为您处理了许多细节。Go在编译时使用逃逸分析 来决定值在内存中的位置。程序运行时通过使用垃圾回收器 跟踪和管理堆分配。虽然在应用程序中创建内存泄漏 不是不可能的,但是这种可能性已经大大降低了。
一种常见的内存泄漏类型就是 Goroutines 泄漏。如果你开始了一个你认为最终会终止但是它永远不会终止的 Goroutine,那么它就会泄露了。它的生命周期为程序的生命周期,任何分配给 Goroutine 的内存都不能释放。所以在这里建议“永远不要在不知道如何停止的情况下,就去开启一个 Goroutine ”。
1 // leak 是一个有 bug 程序。它启动了一个 goroutine
2 // 阻塞接收 channel。一切都将不复存在
3 // 向那个 channel 发送数据,并且那个 channel 永远不会关闭
4 // 那个 goroutine 会被永远锁死
5 func leak() {
6 ch := make(chan int)
7
8 go func() {
9 val := <-ch
10 fmt.Println("We received a value:", val)
11 }()
12 }
1 // serach 函数得到的返回值用 result 结构体来保存
2 // 通过单个 channel 来传递这两个值
3 type result struct {
4 record string
5 err error
6 }
7
8 // process 函数是一个用来寻找记录的函数
9 // 然后打印,如果超过 100 ms 就会失败 .
10 func process(term string) error {
11
12 // 创建一个在 100 ms 内取消上下文的 context
13 ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
14 defer cancel()
15
16 // 为 Goroutine 创建一个传递结果的 channel
17 ch := make(chan result)
18
19 // 启动一个 Goroutine 来寻找记录,然后得到结果
20 // 将返回值从 channel 中返回
21 Go func() {
22 record, err := search(term)
23 ch <- result{record, err}
24 }()
25
26 // 阻塞等待从 Goroutine 接收值
27 // 通过 channel 和 context 来取消上下文操作
28 select {
29 case <-ctx.Done():
30 return errors.New("search canceled")
31 case result := <-ch:
32 if result.err != nil {
33 return result.err
34 }
35 fmt.Println("Received:", result.record)
36 return nil
37 }
38 }
在第 23 行,它通过channel 发送。在此 channel上发送将阻塞执行,直到另一个 Goroutine准备接收值为止。在超时的情况下,接收方停止等待Goroutine 的接收并继续工作。这将导致 Goroutine 永远阻塞,等待一个永远不会发生的接收器出现。这就是 Goroutine 泄露的时候。
修复:创造一些空间,解决此泄漏的最简单方法是将无缓冲 channel 更改为容量为 1 的缓冲通道。
现在我会给你这个建议:
任何时候你开始 Goroutine 你必须问自己:
- 什么时候会终止?
- 什么可以阻止它终止?
并发是一个有用的工具,但必须谨慎使用。
在此内存泄漏示例中,您将看到多个 Goroutines 被阻塞等待接收永远不会发送的值。
文章中程序启动了多个 Goroutines 来处理文件中的一批记录。每个 Goroutine 从输入通道接收值,然后通过输出通道发送新值。
35// processRecords is given a slice of values such as lines
36// from a file. The order of these values is not important
37// so the function can start multiple workers to perform some
38// processing on each record then feed the results back.
39func processRecords(records []string) {
40
41 // Load all of the records into the input channel. It is
42 // buffered with just enough capacity to hold all of the
43 // records so it will not block.
44
45 total := len(records)
46 input := make(chanstring, total)
47 for _, record := range records {
48 input <- record
49 }
50 // close(input) // What if we forget to close the channel?
51
52 // Start a pool of workers to process input and send
53 // results to output. Base the size of the worker pool on
54 // the number of logical CPUs available.
55
56 output := make(chanstring, total)
57 workers := runtime.NumCPU()
58 for i := 0; i < workers; i++ {
59 go worker(i, input, output)
60 }
61
62 // Receive from output the expected number of times. If 10
63 // records went in then 10 will come out.
64
65 for i := 0; i < total; i++ {
66 result := <-output
67 fmt.Printf("[result ]: output %s\n", result)
68 }
69 }
70
71// worker is the work the program wants to do concurrently.
72// This is a blog post so all the workers do is capitalize a
73// string but imagine they are doing something important.
74//
75// Each goroutine can't know how many records it will get so
76// it must use the range keyword to receive in a loop.
77func worker(id int, input <-chan string, output chan<- string) {
78 for v := range input {
79 fmt.Printf("[worker %d]: input %s\n", id, v)
80 output <- strings.ToUpper(v)
81 }
82 fmt.Printf("[worker %d]: shutting down\n", id)
83 }
该程序从未到达第 82 行,该行将宣布程序正在关闭。即使在 processRecords 函数返回之后,每个 worker Goroutines 仍处于活动状态并等待第 78 行的输入。通道会一直接收数据直到通道关闭并为空。问题是程序永远不会关闭通道。
修复泄漏只需要一行代码: close(input) 。关闭频道是表示”不再发送数据“的一种方式。关闭通道的最合适位置是在第 50 行发送最后一个值之后。
重复这一建议
如果不知道它会如何停止,就不要开始使用 goroutine
package main
import (
"fmt"
"time"
)
const (
ANSIC = "Mon Jan _2 15:04:05 2006"
UnixDate = "Mon Jan _2 15:04:05 MST 2006"
RubyDate = "Mon Jan 02 15:04:05 -0700 2006"
RFC822 = "02 Jan 06 15:04 MST"
RFC822Z = "02 Jan 06 15:04 -0700" // RFC822 with numeric zone
RFC850 = "Monday, 02-Jan-06 15:04:05 MST"
RFC1123 = "Mon, 02 Jan 2006 15:04:05 MST"
RFC1123Z = "Mon, 02 Jan 2006 15:04:05 -0700" // RFC1123 with numeric zone
RFC3339 = "2006-01-02T15:04:05Z07:00"
RFC3339Nano = "2006-01-02T15:04:05.999999999Z07:00"
Kitchen = "3:04PM"
// Handy time stamps.
Stamp = "Jan _2 15:04:05"
StampMilli = "Jan _2 15:04:05.000"
StampMicro = "Jan _2 15:04:05.000000"
StampNano = "Jan _2 15:04:05.000000000"
)
func main() {
fmt.Println(time.Now().Format("2006")) // 2020
fmt.Println(time.Now().Format("2006-01")) // 2020-03
fmt.Println(time.Now().Format("2006-01-02")) // 2020-03-03
fmt.Println(time.Now().Format("2006-01-02 15")) // 2020-03-03 04
fmt.Println(time.Now().Format("2006-01-02 15:04")) // 2020-03-03 04:41
fmt.Println(time.Now().Format("2006-01-02 15:04:05")) // 2020-03-03 04:41:05
fmt.Println(time.Now().Format("2006-01-02 15:04:05.000")) // 2020-03-03 04:41:05.721
fmt.Println(time.Now().Format("2006-01-02 15:04:05.000000")) // 2020-03-03 04:41:05.721408
fmt.Println(time.Now().Format("2006-01-02 15:04:05.000000000")) // 2020-03-03 04:41:05.721408664
fmt.Println(time.Now().Format("2006-01-02 15:04:05Z07:00")) // 2020-03-03 04:41:05Z
fmt.Println(time.Now().Local()) // 2020-03-03 04:41:05.721408664 +0000 UTC
}
https://stackoverflow.com/a/45163813
Go uses the reference date 20060102150405
which seems meaningless but actually has a reason, as it's 1 2 3 4 5 6 in the Posix date command:
Mon Jan 2 15:04:05 -0700 MST 2006
0 1 2 3 4 5 6
The timezone is 7 but that sits in the middle, so in the end the format resembles 1 2 3 4 5 7 6
.
https://play.golang.org/p/K5ZpQsHxlfN
只在我计算机上能跑的代码:select-case-default 忘记让出 CPU 的坑
虽然 select-case-default 会随机均匀的尝试每一个 case,但是并不会主动把 CPU 控制权交出去,需要用
runtime.Gosched()
或<-time.After(time.Microsecond)
把 CPU 让出给其他 goroutine。否则,其他的 goroutine 将可能没有机会运行。C# 里的 Application.DoEvents() 也是一样的意思,让别的事件有机会被触发。
How to use Diago to diagnose CPU and memory usage in Go programs
Mar 15, 2020 — 3 minutes read Diago is a visualization tool for profiles and heap snapshots generated with pprof.
It is a stand-alone application displaying a call tree of the CPU usage per functions (or per line of codes in functions) but also capable of displaying heap snapshots.
We’ll see in this short article how to use Diago to get more insight in your Go programs and to help you debug them.
If you know how to use pprof with Go binaries, this chapter won’t add much to what you already know. If you don’t, you will see that it is quite straight-forward.
There are a few ways of starting profiling using pprof, we will focus on the easiest one: using its HTTP handler.
The Go standard library ships an HTTP handler exposing profiling samples, heap snashots, goroutines information, traces and much more. In order to enable it, you have to import net/http/pprof if you’re already using a HTTP server in your program:
import _ "net/http/pprof"
If you are not already serving HTTP in your program, the easiest way to get started is to import the packages as mentioned above and to add this snippet to your program:
go func() {
log.Println(http.ListenAndServe("localhost:6060", nil))
}()
With this done, pprof is exposing all the information on http://localhost:6060/debug/pprof.
Now that pprof is exposing its data through HTTP, it’s quite easy to retrieve profiles and heap snashots.
A profile will contain information on the amount of time spent by the CPU in each function. It is a proportional value: pprof won’t observe exactly how long each function is running on the CPU, instead, pprof is frequently capturing samples of the CPU usage and they are later aggregated by the visualization tools.
Because the samples are frequently captured by pprof, you can decide how long you want pprof to gather them. Let’s say we want to let pprof run for 30s, we will use curl to download a 30s profile:
curl "http://localhost:6060/debug/pprof/profile?seconds=30" > profile
Pprof lets you capture a snapshot of the heap as well. Here’s the command to capture it:
curl "http://localhost:6060/debug/pprof/heap" > heap
We will see in the visualization chapter that there is two kind of information in this file: the total amount of memory allocated per functions since the start of the program and the amount of memory used by each function at the moment of capture.
First, if you want to compile and install Diago, use the command:
go get -u github.com/remeh/diago
In order to visualize the profile, start Diago and use the -file flag to provide the filepath of the file to open:
./diago -file profile
and here you go, you should have a visualization such as this one:
You can do the same to open the heap snapshot:
./diago -file heap
In the screenshot above, you can observe that we’re looking at the total memory allocated by each function of the program since it has started (the observed heap in this screenshot is actually a heap snapshot of Diago).
On the top right, you can see a switch to select the other mode displaying the memory usage at the moment of the capture.
There is other tools helping you visualize these files in flamegraph or in a dot diagram, you can even use the pprof command-line tool to display the data by top values and more.
I’ve developed this tool because I sometimes want to visualize this kind of information in a tree and I hope that it could be useful to you as well.
The Go SSA Playground: A web service for exploring Go's SSA intermediate representation. https://github.com/changkun/ssaplayground
https://wiki.jikexueyuan.com/project/the-go-study-notes-fourth-edition/appendix.html
参数 | 说明 | 示例 |
---|---|---|
-gcflags | 传递给5g/6g/8g编译器的参数(5:arm, 6:x86-64, 8:x86 | |
-ldflags | 传递给5l/6l/8l连接器的参数 | |
-work | 查看编译临时目录 | |
-race | 允许数据竞争检测(仅支持amd64) | |
-n | 查看但不执行编译命令 | |
-x | 查看并执行编译命令 | |
-a | 强制重新编译所有依赖包 | |
-v | 查看被编译的报名,包括依赖包 | |
-p n | 并行编译所使用的CPU core数据。默认全部 | |
-o | 输出文件名 |
参数 | 说明 | 示例 |
---|---|---|
-B | 禁用边界检查 | |
-N | 禁用优化 | |
-l | 禁用函数内联 | |
-u | 禁用unsafe代码 | |
-m | 输出优化信息 | |
-S | 输出汇编代码 |
参数 | 说明 | 示例 |
---|---|---|
-w | 禁用DRAWF调试信息,但不包括符号表 | |
-s | 禁用符号表 | |
-X | 修改字符串符号值 | -X main.VER '0.99' -X main.S 'abc' |
-H | 链接文件类型,其中包括windowsgui | cmd/ld/doc.go |
go tool 6g -h
或 https://golang.org/cmd/gc/go tool 6l -h
或 https://golang.org/cmd/ld/和 go build 参数相同,将生成文件拷贝到 bin、pkg 目录。优先使用 GOBIN 环境变量所指定目录。
参数 | 说明 | 示例 |
---|---|---|
-n | 查看但不执行清理命令 | |
-x | 查看并执行清理命令 | |
-i | 删除bin pkg目录下的二进制文件 | |
-r | 清理所有依赖包临时文件 |
下载并安装扩展包。默认保存到 GOPATH 指定的第一个工作空间。
参数 | 说明 | 示例 |
---|---|---|
-d | 仅下载,不执行安装命令 | |
-t | 下载测试所需的依赖包 | |
-u | 更新包,包括其依赖包 | |
-v | 查看并执行命令 |
反汇编可执行文件
$ go tool objdump -s "main\.\w+" test
$ go tool objdump -s "main\.main" test
通过 runtime.GOOS/GOARCH 判断,或使用编译约束标记。
// +build darwin linux
<--- 必须有空行,以区别包文档。
package main
在源文件 (.go, .h, .c, .s 等) 头部添加 "+build" 注释,指示编译器检查相关环境变量。多个约束标记会合并处理。其中空格表示 OR,逗号 AND,感叹号 NOT。
// +build darwin linux --> 合并结果 (darwin OR linux) AND (amd64 AND (NOT cgo))
// +build amd64,!cgo
如果 GOOS、GOARCH 条件不符合,则编译器会会忽略该文件。
还可使用文件名来表示编译约束,比如 test_darwin_amd64.go。使用文件名拆分多个不同平台源文件,更利于维护。
$ ls -l /usr/local/go/src/pkg/runtime
-rw-r--r--@ 1 yuhen admin 11545 11 29 05:38 os_darwin.c
-rw-r--r--@ 1 yuhen admin 1382 11 29 05:38 os_darwin.h
-rw-r--r--@ 1 yuhen admin 6990 11 29 05:38 os_freebsd.c
-rw-r--r--@ 1 yuhen admin 791 11 29 05:38 os_freebsd.h
-rw-r--r--@ 1 yuhen admin 644 11 29 05:38 os_freebsd_arm.c
-rw-r--r--@ 1 yuhen admin 8624 11 29 05:38 os_linux.c
-rw-r--r--@ 1 yuhen admin 1067 11 29 05:38 os_linux.h
-rw-r--r--@ 1 yuhen admin 861 11 29 05:38 os_linux_386.c
-rw-r--r--@ 1 yuhen admin 2418 11 29 05:38 os_linux_arm.c
支持:*_GOOS、*_GOARCH、*_GOOS_GOARCH、*_GOARCH_GOOS
格式。
可忽略某个文件,或指定编译器版本号。更多信息参考标准库 go/build 文档。
// +build ignore
// +build go1.2 <--- 最低需要 go 1.2 编译。
自定义约束条件,需使用 "go build -tags" 参数。
test.go
// +build beta,debug
package main
func init() {
println("test.go init")
}
输出:
$ go build -tags "debug beta" && ./test
test.go init
$ go build -tags "debug" && ./test
$ go build -tags "debug \!cgo" && ./test
考虑到这一点,我这样扩展了 Peter 的建议:
// +build unit - 不需要任何服务的测试,
// +build integration - 一个强制标记来测试我们自己的服务,
// +build external - 针对第三方和公共服务进行测试,
// +build integration,external - 针对我们自己的服务以及公共服务进行测试,
// +build !integration,external - 专门针对公共服务进行测试,
// +build unit integration - 不依赖服务,提供集成功能
我们的测试通常属于单元测试、集成测试或外部测试的某一类,或者是它们的某种组合。我们肯定希望在 CI 任务中跳过 external 测试,原因显而易见,但如果我们正在考虑调试开发中的一些相关问题,它们是非常有价值的。我们经常需要定位到具体包中具体的测试,因此运行类似下面的内容是有意义的:
go test --tags="integration external" ./messaging/webhooks/...
首先得生成与平台相关的工具和标准库。
$ cd /usr/local/go/src
$ GOOS=linux GOARCH=amd64 ./make.bash --no-clean
# Building C bootstrap tool.
cmd/dist
# Building compilers and Go bootstrap tool for host, darwin/amd64.
cmd/6l
cmd/6a
cmd/6c
cmd/6g
...
---
Installed Go for linux/amd64 in /usr/local/go
Installed commands in /usr/local/go/bin
说明:参数 no-clean 避免清除其他平台文件。 然后回到项目所在目录,设定 GOOS、GOARCH 环境变量即可编译目标平台文件。
$ GOOS=linux GOARCH=amd64 go build -o test
$ file test
learn: ELF 64-bit LSB executable, x86-64, version 1 (SYSV)
$ uname -a
Darwin Kernel Version 12.5.0: RELEASE_X86_64 x86_64
简单点说,go generate 扫描源代码文件,找出所有 "//go:generate" 注释,提取并执行预处理命令。
不属于 build 组成部分,设计目标是提供给包开发者使用,因为包用户可能不具备命令执行环境。
//go:generate ls -l
//go:generate du
参数 | 说明 | 示例 |
---|---|---|
-x | 显示并执行命令 | |
-n | 显示但不执行命令 | |
-v | 输出处理的包和源文件名 |
还可定义别名。须提前定义,仅在当前文件内有效。
//go:generate -command YACC go tool yacc
//go:generate YACC -o test.go -p parse test.y
可用条件编译,让 go build 忽略包含 generate 的文件。
// +build generate
$ go generate -tags generate
资源:
默认情况下,编译的二进制文件已包含 DWARFv3 调试信息,只要 GDB 7.1 以上版本都可以调试。
相关选项:
除了使用 GDB 的断点命令外,还可以使用 runtime.Breakpoint 函数触发中断。另外,runtime/debug.PrintStack 可用来输出调用堆栈信息。
某些时候,需要手工载入 Go Runtime support (runtime-gdb.py)。
.gdbinit
define goruntime
source /usr/local/go/src/runtime/runtime-gdb.py
end
set disassembly-flavor intel
set print pretty on
dir /usr/local/go/src/pkg/runtime
数据竞争 (data race) 是并发程序里不太容易发现的错误,且很难捕获和恢复错误现场。Go 运行时内置了竞争检测,允许我们使用编译器参数打开这个功能。它会记录和监测运行时内存访问状态,发出非同步访问警告信息。
func main() {
var wg sync.WaitGroup
wg.Add(2)
x := 100
go func() {
defer wg.Done()
for {
x += 1
}
}()
go func() {
defer wg.Done()
for {
x += 100
}
}()
wg.Wait()
}
输出:
$ GOMAXPROCS=2 go run -race main.go
==================
WARNING: DATA RACE
Write by goroutine 4:
main.func·002()
main.go:25 +0x59
Previous write by goroutine 3:
main.func·001()
main.go:18 +0x59
Goroutine 4 (running) created at:
main.main()
main.go:27 +0x16f
Goroutine 3 (running) created at:
main.main()
main.go:20 +0x100
==================
数据竞争检测会严重影响性能,不建议在生产环境中使用。
func main() {
x := 100
for i := 0; i < 10000; i++ {
x += 1
}
fmt.Println(x)
}
输出:
$ go build && time ./test
10100
real" 0m0.060s
user" 0m0.001s
sys" 0m0.003s
$ go build -race && time ./test
10100
real" 0m1.025s
user" 0m0.003s
sys" 0m0.009s
通常作为非性能测试项启用。
$ go test -race
自带代码测试、性能测试、覆盖率测试框架。
注: 不要将代码放在名为 main 的目录下,这会导致 go test "cannot import main" 错误。
testing.T
方法|说明|其它 Fail|标记失败,但继续执行该测试函数| FailNow|失败,立即停止当前测试函数| Log|输出信息。尽在失败或者-v参数时输出|Logf SkipNow|跳过当前测试函数 | Skipf = SkipNow + Logf Error | Fail + Log | Errorf Fatal | FailNow + Log | Fatalf
main_test.go
package main
import (
"testing"
"time"
)
func sum(n ...int) int {
var c int
for _, i := range n {
c += i
}
return c
}
func TestSum(t *testing.T) {
time.Sleep(time.Second * 2)
if sum(1, 2, 3) != 6 {
t.Fatal("sum error!")
}
}
func TestTimeout(t *testing.T) {
time.Sleep(time.Second * 5)
}
默认 go test 执行所有单元测试函数,支持 go build 参数。
参数 | 说明 | 示例 |
---|---|---|
-c | 仅编译,不执行测试 | |
-v | 显示所有测试函数执行细节 | |
-run regex | 执行指定的测试函数(正则表达式) | |
-parallel n | 并发执行测试函数(默认: GOMAXPROCS) | |
-timeout t | 单个测试超时时间 | -timeout 2m30s |
$ go test -v -timeout 3s
=== RUN TestSum
--- PASS: TestSum (2.00 seconds)
=== RUN TestTimeout
panic: test timed out after 3s
FAIL" test" 3.044s
$ go test -v -run "(i)sum"
=== RUN TestSum
--- PASS: TestSum (2.00 seconds)
PASS
ok " test" 2.044s
可重写 TestMain 函数,处理一些 setup/teardown 操作。
func TestMain(m *testing.M) {
println("setup")
code := m.Run()
println("teardown")
os.Exit(code)
}
func TestA(t *testing.T) {}
func TestB(t *testing.T) {}
func BenchmarkC(b *testing.B) {}
输出:
$ go test -v -test.bench .
setup
=== RUN TestA
--- PASS: TestA (0.00s)
=== RUN TestB
--- PASS: TestB (0.00s)
PASS
BenchmarkC" 2000000000" 0.00 ns/op
teardown
ok " test" 0.028s
性能测试需要运行足够多的次数才能计算单次执行平均时间。
func BenchmarkSum(b *testing.B) {
for i := 0; i < b.N; i++ {
if sum(1, 2, 3) != 6 {
b.Fatal("sum")
}
}
}
默认情况下,go test 不会执行性能测试函数,须使用 "-bench" 参数。
go test
参数 | 说明 | 示例 |
---|---|---|
-bench regex | 执行指定性能测试函数(正则表达式) | |
-benchmem | 输出内存统计信息 | |
-benchtime -t | 设置每个性能测试运行时间 | -benchtime 1m30s |
-cpu | 设置并发测试。 默认 GOMAXPROCS | -cpu 1,2,4 |
$ go test -v -bench .
=== RUN TestSum
--- PASS: TestSum (2.00 seconds)
=== RUN TestTimeout
--- PASS: TestTimeout (5.00 seconds)
PASS
BenchmarkSum 100000000 11.0 ns/op
ok " test" 8.358s
$ go test -bench . -benchmem -cpu 1,2,4 -benchtime 30s
BenchmarkSum 5000000000 11.1 ns/op 0 B/op 0 allocs/op
BenchmarkSum-2 5000000000 11.4 ns/op 0 B/op 0 allocs/op
BenchmarkSum-4 5000000000 11.3 ns/op 0 B/op 0 allocs/op
ok " test" 193.246s
与 testing.T 类似,区别在于通过捕获 stdout 输出来判断测试结果。
func ExampleSum() {
fmt.Println(sum(1, 2, 3))
fmt.Println(sum(10, 20, 30))
// Output:
// 6
// 60
}
不能使用内置函数 print/println,它们默认输出到 stderr。
$ go test -v
=== RUN: ExampleSum
--- PASS: ExampleSum (8.058us)
PASS
ok " test" 0.271s
Example 代码可输出到文档,详情参考包文档章节。
除显示代码覆盖率百分比外,还可输出详细分析记录文件。
参数 | 说明 |
---|---|
-cover | 允许覆盖分析 |
-covermode | 代码分析模式(set: 是否执行;count:执行次数; atomic: 次数,并发支持) |
-coverprofile | 输出结果文件 |
$ go test -cover -coverprofile=cover.out -covermode=count
PASS
coverage: 80.0% of statements
ok " test" 0.043s
$ go tool cover -func=cover.out
test.go: Sum 100.0%
test.go: Add 0.0%
total:" (statements) 80.0%
用浏览器输出结果,能查看更详细直观的信息。包括用不同颜色标记覆盖、运行次数等。
$ go tool cover -html=cover.out
说明:将鼠标移到代码块,可以看到具体的执行次数。
监控程序执行,找出性能瓶颈。
import (
"os"
"runtime/pprof"
)
func main() {
// CPU
cpu, _ := os.Create("cpu.out")
defer cpu.Close()
pprof.StartCPUProfile(cpu)
defer pprof.StopCPUProfile()
// Memory
mem, _ := os.Create("mem.out")
defer mem.Close()
defer pprof.WriteHeapProfile(mem)
}
除调用 runtime/pprof 相关函数外,还可直接用测试命令输出所需记录文件。
参数|说明 -blockprofile block.out|goroutine阻塞 -blockprofilerate n | 超出该参数设置时间 的阻塞才被记录,单位:纳秒 -cpuprofile cpu.out | CPU -memprofile mem.out | 内存分配 -memprofilerate n | 超出该参数(bytes)设置的内存分配才被记录,默认 512KB (mprof.go)
以 net/http 包为演示,先生成记录文件。
$ go test -v -test.bench "." -cpuprofile cpu.out -memprofile mem.out net/http
进入交互式查看模式。
$ go tool pprof http.test mem.out
(pprof) top5
2597.58kB of 2597.58kB total ( 100%)
Dropped 421 nodes (cum <= 12.99kB)
Showing top 5 nodes out of 28 (cum >= 1536.60kB)
flat flat% sum% cum cum%
1024.04kB 39.42% 39.42% 1024.04kB 39.42% encoding/asn1.parsePrintableString
548.84kB 21.13% 60.55% 548.84kB 21.13% mime.setExtensionType
512.56kB 19.73% 80.28% 1536.60kB 59.16% crypto/x509.parseCertificate
512.14kB 19.72% 100% 512.14kB 19.72% mcommoninit
0 0% 100% 1536.60kB 59.16% crypto/tls.(*Conn).Handshake
默认输出 inuse_space,可在命令行指定其他值,包括排序方式。
$ go tool pprof -alloc_space -cum http.test mem.out
可输出函数调用的列表统计信息。
或者是更详细的源码模式。
除交互模式外,还可直接输出统计结果。
$ go tool pprof -text http.test mem.out
2597.58kB of 2597.58kB total ( 100%)
Dropped 421 nodes (cum <= 12.99kB)
flat flat% sum% cum cum%
1024.04kB 39.42% 39.42% 1024.04kB 39.42% encoding/asn1.parsePrintableString
548.84kB 21.13% 60.55% 548.84kB 21.13% mime.setExtensionType
512.56kB 19.73% 80.28% 1536.60kB 59.16% crypto/x509.parseCertificate
512.14kB 19.72% 100% 512.14kB 19.72% mcommoninit
0 0% 100% 1536.60kB 59.16% crypto/tls.(*Conn).Handshake
0 0% 100% 1536.60kB 59.16% crypto/tls.(*Conn).clientHandshake
输出图形文件。
$ go tool pprof -web http.test mem.out
还可用 net/http/pprof 实时查看 runtime profiling 信息。
package main
import (
_ "net/http/pprof"
"net/http"
"time"
)
func main() {
go http.ListenAndServe("localhost:6060", nil)
for {
time.Sleep(time.Second)
}
}
在浏览器中查看 http://localhost:6060/debug/pprof/ 。
附: 自定义统计数据,可用 expvar 导出,用浏览器访问 /debug/vars 查看。
Find out the version of Go a Binary was built with?
➜ gonginx git:(master) go install ./... [一 6/15 15:13:46 2020]
➜ gonginx git:(master) strings ~/go/bin/gonginx | grep 'Go build\|go1\.' [一 6/15 15:13:49 2020]
stack=[address bad instboundarycgocheckcontinuedefault:durationencodingexporterfc00::/7filenamefinishedfont/otffont/ttfgo1.14.2hijackedhttp/1.1https://if-matchif-rangeinfinityinvalid locationloopbackno-cacheno_proxyopPseudooverflowraw-readreadfromrecvfromreflect.runnableruntime.scavengeshutdownstrconv.templatethetasymtimeout:truncateunixgramunknown(urlquery (forced) -> node= as type blocked= defersc= in use)
/usr/local/go/src/vendor/golang.org/x/crypto/poly1305/bits_go1.13.go
Go buildinf:
➜ gonginx git:(master) go install -ldflags="-s -w" ./... [一 6/15 15:13:50 2020]
➜ gonginx git:(master) strings ~/go/bin/gonginx | grep 'Go build\|go1\.' [一 6/15 15:13:55 2020]
stack=[address bad instboundarycgocheckcontinuedefault:durationencodingexporterfc00::/7filenamefinishedfont/otffont/ttfgo1.14.2hijackedhttp/1.1https://if-matchif-rangeinfinityinvalid locationloopbackno-cacheno_proxyopPseudooverflowraw-readreadfromrecvfromreflect.runnableruntime.scavengeshutdownstrconv.templatethetasymtimeout:truncateunixgramunknown(urlquery (forced) -> node= as type blocked= defersc= in use)
/usr/local/go/src/vendor/golang.org/x/crypto/poly1305/bits_go1.13.go
Go buildinf:
➜ gonginx git:(master) go install ./... && upx --best --lzma ~/go/bin/gonginx [一 6/15 15:14:15 2020]
Ultimate Packer for eXecutables
Copyright (C) 1996 - 2018
UPX 3.95 Markus Oberhumer, Laszlo Molnar & John Reiser Aug 26th 2018
File size Ratio Format Name
-------------------- ------ ----------- -----------
11508820 -> 5824528 50.61% macho/amd64 gonginx
Packed 1 file.
➜ gonginx git:(master) strings ~/go/bin/gonginx | grep 'Go build\|go1\.' [一 6/15 15:14:26 2020]
Go buildinf:
➜ gonginx git:(master) go install -ldflags="-s -w" ./... && upx ~/go/bin/gonginx [一 6/15 15:14:28 2020]
Ultimate Packer for eXecutables
Copyright (C) 1996 - 2018
UPX 3.95 Markus Oberhumer, Laszlo Molnar & John Reiser Aug 26th 2018
File size Ratio Format Name
-------------------- ------ ----------- -----------
9182292 -> 3493904 38.05% macho/amd64 gonginx
Packed 1 file.
➜ gonginx git:(master) strings ~/go/bin/gonginx | grep 'Go build\|go1\.' [一 6/15 15:14:44 2020]
Go build ID: "MokNR
Go buildinf:
➜ gonginx git:(master) strings ~/go/bin/gonginx | grep 'UPX\|Go build\|go1\.' [一 6/15 15:23:12 2020]
UPX!
Go build ID: "MokNR
Go buildinf:
UPX!
$Info: This file is packed with the UPX executable packer http://upx.sf.net $
$Id: UPX 3.95 Copyright (C) 1996-2018 the UPX Team. All Rights Reserved. $
UPX!t
UPX!
from blog Go with Examples: Keeping Your Modules Compatible
Struct values are comparable if all their fields are comparable. Two struct values are equal if their corresponding non-blank fields are equal.
In other words, if you add on a slice, map or function field into a struct, we will not be able to perform a comparison on the struct.
https://play.golang.org/p/j533RPDkOSo
package main
import (
"fmt"
)
// OK!
type ComparableStruct struct {
i int
s string
b bool
}
// Unable to perform == or != operation on this struct
type UncomparableStruct struct {
i int
s string
b bool
v []int
}
func main() {
comparable1 := ComparableStruct{1,"s", true}
comparable2 := ComparableStruct{1,"s", true}
uncomparable1 := UncomparableStruct{1,"s", true, []int{1,2,3}}
uncomparable2 := UncomparableStruct{1,"s", true, []int{1,2,3}}
fmt.Println(comparable1 == comparable2)
fmt.Println(uncomparable1 == uncomparable2) //error not comparable
// ./prog.go:28:30: invalid operation: uncomparable1 == uncomparable2 (struct containing []int cannot be compared)
}
When an Interface Depends on Another Interface in Go
textworker/textworker.go
package textworker
import "fmt"
type Worker struct {}
func (w Worker) Do() {
fmt.Println("I am working")
}
async/asyncrunner.go
package async
type Worker interface {
Do()
}
type Runner struct{}
func (r Runner) Run(w Worker) {
go w.Do()
}
main.go
package main
import (
"async"
"runner"
"textworker"
)
func main() {
worker := textworker.Worker{}
asyncRunner := async.Runner{}
runner.Run(asyncRunner, worker) // compiler error
}
🕙[2020-09-02 09:47:42.838] ❯ go build
# interfacedep
./main.go:13:12: cannot use asyncRunner (type async.Runner) as type runner.Runner in argument to runner.Run:
async.Runner does not implement runner.Runner (wrong type for Run method)
have Run(async.Worker)
want Run(runner.Worker)
改进方法1:
type Runner interface {
Run(workerFunc func())
}
改进方法2:
type Runner interface {
Run(w interface{
Do()
})
}
改进方法2.1,匿名接口内嵌Worker
type Runner interface {
Run(w interface{Worker})
}
Example of cross-package interfaces in golang上的例子
test_a_sayer.go
package a
import "fmt"
type Sayer interface {
Say() string
}
type Formal struct{}
func (p Formal) Greet(s interface{Sayer}) string {
return fmt.Sprintf("Hello, %s", s.Say())
}
test_b_sayer.go:
package b
import "fmt"
type Sayer interface {
Say() string
}
type Casual struct{}
func (p Casual) Greet(s interface{Sayer}) string {
return fmt.Sprintf("Hey %s!", s.Say())
}
test_greet_greet.go:
package greet
type Sayer interface {
Say() string
}
type Greeter interface {
Greet(s interface{Sayer}) string
}
func Greet(g interface{Greeter}, i Sayer) string {
return g.Greet(i)
}
test_main.go:
package main
import (
"log"
"test/a"
"test/b"
"test/greet"
)
type Who struct {
to string
}
func (w Who) Say() string {
return w.to
}
type Bad struct {
to string
}
func (w Bad) Say(exclaim bool) string {
said := w.to
if exclaim {
said += "!!!"
}
return said
}
func main() {
log.Println("Testing...")
a := a.Formal{}
b := b.Casual{}
w := &Who{
to:"world",
}
x := &Bad{
to:"world",
}
log.Println(greet.Greet(a, w))
log.Println(greet.Greet(b, w))
//log.Println(greet.Greet(b, x))
}
并发bug发现不易的一个例子
FINDING GOROUTINE BUGS WITH TLA+
https://github.com/google/gops/blob/5d514cabbb21de80cb27f529555ee090b60bbb80/goprocess/gp.go#L29
/*1 */ func FindAll() []P { //P = process data
/*2 */ pss, err := ps.Processes()
/*3 */ [...]
/*4 */ found := make(chan P)
/*5 */ limitCh := make(chan struct{}, concurrencyProcesses)
/*6 */
/*7 */ for _, pr := range pss {
/*8 */ limitCh <- struct{}{}
/*9 */ pr := pr
/*10*/ go func() {
/*11*/ defer func() { <-limitCh }()
/*12*/ [... get a P with some error checking ...]
/*13*/ found <- P
/*14*/ }()
/*15*/ }
/*16*/ [...]
/*17*/
/*18*/ var results []P
/*19*/ for p := range found {
/*20*/ results = append(results, p)
/*21*/ }
/*22*/ return results
/*23*/ }
以上代码,可能会出现死锁。其实有两个并发的bug。
Even in Go, concurrency is still not easy (with an example) September 1, 2020 Go is famous for making concurrency easy, through good language support for goroutines. Except what Go makes easy is only one level of concurrency, the nuts and bolts level of making your code do things concurrently and communicating back and forth through channels. Making it do the right things concurrently is still up to you, and unfortunately Go doesn't currently provide a lot of standard library support for correctly implemented standard concurrency patterns.
For example, one common need is for a limited amount of concurrency; you want to do several things at once, but only so many of them. At the moment this is up to you to implement on top of goroutines, channels, and things like the sync package. This is not as easy as it looks, and quite competent people can make mistakes here. As it happens, I have an example ready to hand today.
Gops is a convenient command to list (and diagnose) Go processes that are currently running on your system. Among other things, it'll tell you which version of Go they were compiled with, which is handy if you want to see if you have out of date binaries that should be rebuilt and redeployed. One of the things gops needs to do is look at all of the Go processes on your system, which it does concurrently. However, it doesn't want to look at too many processes at once, because that can cause problems with file descriptor limits. This is a classic case of limited concurrency.
Gops implements this at the moment with code in goprocess.FindAll() that looks like this, in somewhat sketched and reduced form:
func FindAll() []P { pss, err := ps.Processes() [...] found := make(chan P) limitCh := make(chan struct{}, concurrencyProcesses)
for _, pr := range pss { limitCh <- struct{}{} pr := pr go func() { defer func() { <-limitCh }() [... get a P with some error checking ...] found <- P }() } [...]
var results []P for p := range found { results = append(results, p) } return results } (In the real code there's a WaitGroup for coordination, and the found channel gets closed appropriately.)
How this works is clear, and is a standard pattern (covered in eg Go 101's Channel Use Cases). We use a buffered channel to provide a limited number of tokens; sending a value into the channel implicitly takes a token (and blocks if the token supply is exhausted), while receiving a value from it puts a token back in. We take a token before we start a new goroutine, and the goroutine releases the token when it's done.
Except that this code has a bug if there are too many processes to examine. Even knowing that there is a bug in this code, it may not be obvious.
The bug is that the goroutines only receive from limitCh to release their token after sending their result to the unbuffered found channel, while the main code only starts receiving from found after running through the entire loop, and the main code takes the token in the loop and blocks if no tokens are available. So if you have too many processes to go through, you start N goroutines, they all block trying to write to found and don't receive from limitCh, and the main for loop blocks trying to send to limitCh and never reaches the point where it starts receiving from found.
Concurrency in its simplest form can be defined as doing multiple things at once. Imagine writing two letters with both hands when you have one pencil, you write a sentence with your right hand on one letter, then write a sentence with your left hand on other letter. You might keep alternating hands, until both the letters are complete. — Doing multiple tasks, by sharing time, resources etc.
Parallelism means starting the tasks, completing the tasks together. Imagine writing the letters with both hands with a pencil in each hand without interruption. This would require a lot of brain power and practice. — Doing multiple tasks “at the same time”
我的看法:明天将要发布的代码选择 Go,在未来五年内保持不变的代码选择 Rust。—Grzegorz Nosek
如果你想加快开发速度,也许是因为你要编写许多不同的服务,或者你有庞大的开发团队,那么 Go 是你选择的语言。Go 为你提供了一流的并发性,并且不容许不安全的内存访问(Rust 也不容忍),但不会强迫你管理每个最后的细节。Go 是快速而强大的工具,但是它避免了使开发人员陷入困境,而专注于简单性和统一性。另一方面,如果需要拧紧块性能,那么 Rust 应该是你的选择。—Andrew Lader[23]
Find your leaf packages loov/goda:GoDependencyAnalysistoolkit
go get github.com/loov/goda
goda graph ./... | dot -Tsvg -o graph.svg
go get github.com/alvaroloes/enumer
enumer -type=Pill -json -transform=snake
有一种未经证实的说法:
Go诞生于C++程序的漫长构建过程中。
如果C++编译很快,那么Robert Griesemer、Rob Pike和Ken Thompson这三位大佬也没有闲暇时间一起喝着咖啡并决定是时候设计一门新语言了。的确,Go语言诞生后,其简洁的语法、极速地构建、新颖的并发结构、体验优良的工具链以及完成度不低的标准库吸引了很多C/C++程序员转型成为Gopher并开始重度使用Go
回到 2007 年 9 月,我在一个巨大的 Google C++ 程序(就是你们都用过的那个)上做一些琐碎但是很核心的工作,我在那个巨大的分布式集群上需要花大约 45 分钟进行编译。收到一个通知说 Google 雇佣的一对为 C++ 标准化委员会工作的夫妇将会做一场报告。收到一个通知说几个受雇于 Google 的为 C++ 标准化委员会工作的人将会做一场报告。他们将向我们介绍那时还被称作 C++0x(就是现在众所周知的 C++11)中将会有哪些改进。
在长达一个小时的报告中,我们听说了诸如有已经在计划中的 35 个特性之类的事情。事实上有更多,但仅有 35 个特性在报告中进行了描述。当然一些特性很小,但是意义重大,值得在报告中提出。一些非常微妙和难以理解,如左右值引用(rvalue references),还有一些是 C++ 特有的,如可变参数模板(variadic templates),还有一些就是发疯,如用户定义数据标识(user-defined literals)。
这时我问了自己一个问题:C++ 委员会真得相信 C++ 的问题在于没有足够的特性?肯定的说,在另一个 Ron Hardin 的玩笑中,简化语言的成就远远大于添加功能。当然这有点可笑,不过请务必记住这个思路。
There are three different threading models we can see Mx1, 1x1, MxN
FROM Different Threading Models — Why I Feel Go Threading Is Better
应该有很多人受不了err的反复判断,封装各种err处理,达到简化的目的,其中一种实现ErrorFlow Declarative error handling for Go.
func GzipFile(dstFilename string, srcFilename string) (err error) {
// defer IfError()... creates and configures
// ErrorFlow error handler for this function.
// When any of Check* functions encounters non-nil error
// it immediately sends error to this handler
// unwinding all stacked defers.
errWrapper := errf.WrapperFmtErrorw("error compressing file")
defer errf.IfError().ReturnFirst().LogIfSuppressed().Apply(errWrapper).ThenAssignTo(&err)
errf.CheckCondition(len(dstFilename) == 0, "dst file should be specified")
errf.CheckCondition(len(srcFilename) == 0, "src file should be specified")
reader := errf.Io.CheckReadCloser(os.Open(srcFilename))
defer errf.With(errWrapper).Log(reader.Close())
writer := errf.Io.CheckWriteCloser(os.Create(dstFilename))
defer errf.Handle().OnAnyErrOrPanic(func() { os.Remove(dstFilename) })
defer errf.CheckErr(writer.Close())
gzipWriter := gzip.NewWriter(writer)
defer errf.CheckErr(gzipWriter.Close())
return errf.CheckDiscard(io.Copy(gzipWriter, reader))
}
开发机上
$ cd app
$ go mod download -v
$ go mod vendor
$ cd ..
$ tar --exclude .git --exclude .idea -czf app.tar.gz app
编译机上
$ tar zxf app.tar.gz
$ cd app
$ go build -mod vendor -o app -ldflags=' -w -s '
package main
import (
_ "expvar"
"net/http"
)
func main() {
http.ListenAndServe(":8100", nil)
}
go install github.com/divan/expvarmon@latest
expvarmon -ports="8100"
Serve embedded filesystem from root path of URL
package main
import (
"embed"
"io/fs"
"log"
"net/http"
)
//go:embed static
var embeddedFS embed.FS
func main() {
serverRoot, err := fs.Sub(embeddedFS, "static")
if err != nil {
log.Fatal(err)
}
http.Handle("/", http.FileServer(http.FS(serverRoot)))
log.Fatal(http.ListenAndServe(":8080", nil))
}
The redress software is a tool for analyzing stripped Go binaries compiled with the Go compiler. It extracts data from the binary and uses it to reconstruct symbols and performs analysis. It essentially tries to "re-dress" a "stripped" binary.
🕙[2021-05-05 22:16:16.837] ❯ redress -help
Usage of redress:
-compiler
Print information
-filepath
Include file path for packages
-force-version string
Forcing and using the given version when analyzing
-interface
Print interfaces
-method
Print type's methods
-pkg
List packages
-src
Print source tree
-std
Include standard library packages
-struct
Print structs
-type
Print all type information
-unknown
Include unknown packages
-vendor
Include vendor packages
-version
Print redress version
🕙[2021-05-05 22:15:51.625] ❯ go install -ldflags="-s -w" ./...
🕙[2021-05-05 22:16:01.530] ❯ ls -lh ~/go/bin/shorturl
-rwxr-xr-x 1 bingoo staff 15M 5 5 22:16 /Users/bingoo/go/bin/shorturl
🕙[2021-05-05 22:15:22.711] ❯ redress -src ~/go/bin/shorturl
Package main: /Users/bingoo/GitHub/shorturl
File: main.go
main Lines: 12 to 22 (10)
Package github.com/bingoohuang/shorturl/pkg: /Users/bingoo/GitHub/shorturl/pkg
File: <autogenerated>
(*noCache)DeactivateURL Lines: 1 to 1 (0)
(*noCache)SavePopularURL Lines: 1 to 1 (0)
(*RedisCache)DeactivateURL Lines: 1 to 1 (0)
(*noCache)LookupURL Lines: 1 to 1 (0)
(*RedisCache)SavePopularURL Lines: 1 to 1 (0)
(*Body)Merge Lines: 1 to 1 (0)
(*URLInput)ValidateExpiry Lines: 1 to 1 (0)
(*RedisCache)LookupURL Lines: 1 to 1 (0)
(*URL)IsActive Lines: 1 to 1 (0)
(*URLInput)GetExpiresOn Lines: 1 to 1 (0)
File: admin.go
ListURLsFilteredFromRequest Lines: 9 to 21 (12)
ListURLsFiltered Lines: 21 to 46 (25)
DeleteURLFromRequest Lines: 46 to 53 (7)
DeleteURLByShortCode Lines: 53 to 65 (12)
File: client.go
CreateURLShortCodeFromRequest Lines: 13 to 25 (12)
CreateURLShortCode Lines: 25 to 55 (30)
LookupOriginURL Lines: 55 to 76 (21)
IncrementHits Lines: 76 to 87 (11)
ValidateURLInput Lines: 87 to 115 (28)
getUniqueShortCode Lines: 115 to 133 (18)
isShortCodeAvail Lines: 133 to 143 (10)
getShortCodeByOriginURL Lines: 143 to 153 (10)
mapKeywords Lines: 153 to 162 (9)
File: config.go
init Lines: 28 to 28 (0)
createDB Lines: 49 to 74 (25)
init0 Lines: 74 to 90 (16)
init.0func1 Lines: 75 to 79 (4)
createEnvName Lines: 90 to 100 (10)
File: controller.go
ListURLs Lines: 10 to 22 (12)
DeleteShortURL Lines: 22 to 33 (11)
CreateShortURL Lines: 33 to 47 (14)
NotFound Lines: 47 to 53 (6)
ServeShortURL Lines: 53 to 68 (15)
File: model.go
URLIsActive Lines: 55 to 100 (45)
(*URLInput)Validate Lines: 100 to 142 (42)
URLInputValidateExpiry Lines: 142 to 164 (22)
URLInputGetExpiresOn Lines: 164 to 173 (9)
URLFilterGetOffset Lines: 173 to 183 (10)
File: redis.go
noCacheLookupURL Lines: 19 to 20 (1)
noCacheDeactivateURL Lines: 20 to 21 (1)
noCacheSavePopularURL Lines: 21 to 29 (8)
CreateRedisCache Lines: 29 to 58 (29)
CreateRedisCachefunc1 Lines: 36 to 37 (1)
RedisCacheLookupURL Lines: 58 to 110 (52)
RedisCacheDeactivateURL Lines: 82 to 93 (11)
RedisCacheSavePopularURL Lines: 93 to 118 (25)
hasURL Lines: 104 to 110 (6)
File: router.go
init1 Lines: 15 to 22 (7)
AssetsRewrite Lines: 22 to 44 (22)
AssetsRewritefunc1 Lines: 23 to 67 (44)
glob.func1 Lines: 34 to 36 (2)
locateHandler Lines: 44 to 66 (22)
RegisterHandlers Lines: 66 to 81 (15)
RegisterHandlersfunc1 Lines: 67 to 83 (16)
Recover Lines: 81 to 94 (13)
Recoverfunc1 Lines: 82 to 95 (13)
Recover.func11 Lines: 83 to 85 (2)
AdminAuth Lines: 94 to 107 (13)
AdminAuthfunc1 Lines: 95 to 102 (7)
validateAdminToken Lines: 107 to 122 (15)
File: util.go
RandomString Lines: 17 to 33 (16)
BodyMerge Lines: 33 to 43 (10)
JSON Lines: 43 to 49 (6)
2019年的一篇文章:https://medium.com/a-journey-with-go/go-keeping-a-variable-alive-c28e3633673a
package main
import (
"io/ioutil"
"log"
"os"
"runtime"
"syscall"
)
type File struct{ d int }
func main() {
file, err := ioutil.TempFile("", "keepalive")
if err != nil {
log.Fatal(err)
}
file.Write([]byte("keepalive"))
file.Close()
defer os.Remove(file.Name())
p := openFile(file.Name())
content := readFile(p.d)
// Ensure p is not finalized until Read returns
// runtime.KeepAlive(p)
println("Here is the content: " + content)
}
func openFile(path string) *File {
d, err := syscall.Open(path, syscall.O_RDONLY, 0)
if err != nil {
panic(err)
}
p := &File{d}
runtime.SetFinalizer(p, func(p *File) {
syscall.Close(p.d)
})
return p
}
func readFile(descriptor int) string {
doSomeAllocation()
var buf [1000]byte
_, err := syscall.Read(descriptor, buf[:])
if err != nil {
panic(err)
}
return string(buf[:])
}
func doSomeAllocation() {
var a *int
// memory increase to force the GC
for i := 0; i < 10000000; i++ {
i := 1
a = &i
}
_ = a
}
输出:
panic: device not configured
goroutine 1 [running]:
main.readFile(0x3, 0x43, 0xc000120010)
/Users/bingoo/GitHub/gogotcha/cmd/keepalive/main.go:51 +0x138
main.main()
/Users/bingoo/GitHub/gogotcha/cmd/keepalive/main.go:23 +0x176
加上runtime.KeepAlive
后(放开对应的注释行),
Here is the content: keepalive
50 度灰,先看书后观影
《五十度灰》(英语:Fifty Shades of Grey)是一部2015年上映的美国情色爱情电影,根据EL·詹姆丝的同名小说改编而成。由萨姆·泰勒-约翰逊执导,凯利·马塞尔编剧,并由达科塔·约翰逊、詹米·多南、珍妮佛·艾莉以及马西雅·盖·哈登主演[6]。电影于2015年2月11日在柏林国际电影节上首映[7],2月13日正式公映。票房获得即时的佳绩,打破了多个票房纪录,全球票房5.71亿美元。
type Foo struct {
Base // 继承
...
}
type Foo struct {
*Base // 虚拟继承
...
}
来源,许式伟, Go vs. GoPlus(Go+) 2021-6-27 北京 GopherChina2021
介绍了 go 中dig和wire两个DI工具。其中dig是通过运行时反射实现的依赖注入。 而wire是根据自定义的代码,通过命令,生成相应的依赖注入代码,在编译期就完成依赖注入,无需反射机制。 这样的好处是:
- 方便排查,如果存在依赖错误,编译时就能发现。而 dig 只能在运行时才能发现依赖错误。
- 避免依赖膨胀,wire生成的代码只包含被依赖的,而dig可能会存在好多无用依赖。
- 依赖关系静态存在源码,便于工具分析。
原因可能会有很多,关于Go语言的特性、优势等。但是最主要的原因,应该是基于两方面的考虑:执行性能&开发效率。
Go 语言以其接近 C 的执行性能和近解释型语言的开发效率,以及近乎于完美的编译速度,已经广泛应用于人工智能、云计算开发、容器虚拟化、大数据开发、 数据分析及科学计算、运维开发、爬虫、游戏开发等领域。
Go语言适用于各个领域行业-前景广阔
看到一种 go 代码,接口里定义一个私有无参无返回值的方法,感觉是一种惯用法,原来是密封接口,保证这个接口不给外部去实现。
// Statement represents a statement.
type Statement interface {
iStatement()
SQLNode
}
func (*Union) iStatement() {}
func (*Select) iStatement() {}
func (*Insert) iStatement() {}
func (*Update) iStatement() {}
func (*Delete) iStatement() {}
func (*Set) iStatement() {}
func (*DDL) iStatement() {}
func (*Show) iStatement() {}
func (*Use) iStatement() {}
func (*OtherRead) iStatement() {}
func (*OtherAdmin) iStatement() {}
func (*TruncateTable) iStatement() {}
同样的
// SelectStatement any SELECT statement.
type SelectStatement interface {
iSelectStatement()
iStatement()
iInsertRows()
AddOrder(*Order)
SetLimit(*Limit)
SQLNode
}
func (*Select) iSelectStatement() {}
func (*Union) iSelectStatement() {}
func (*ParenSelect) iSelectStatement() {}
在 go-profiler-notes 上看到一句话: All models are wrong
本质上,所有模型都是错误的,但有些模型是有用的。
--- Box,乔治·EP;Norman R.Draper(1987)。经验模型的建立和响应面,p。424,威利。ISBN 0471810339。
我认为最好通过两部分来分析它的含义:
“所有模型都是错误的”,也就是说,每个模型都是错误的,因为它是对现实的简化。一些模型,特别是在“硬”科学中,只是有点错误。他们无视摩擦或微小物体的引力作用。其他模型有很多错误-他们忽略了更大的事情。在社会科学中,我们忽略了很多。
“但是有些有用”-简化现实可能非常有用。它们可以帮助我们解释,预测和理解宇宙及其所有组成部分。
这不仅在统计上是正确的!地图是一种模型。他们错了。但是好的地图非常有用。其他有用但错误的模型的例子比比皆是。
比例为1:1的完美地图的幻想已被许多作者使用,包括Lewis Carroll,Jorge Luis Borges和Umberto Eco。实际上,这没有用,因为它所映射的区域必然很复杂,而且不容易理解(更不用说将其展开并布置为阅读的尴尬了)。
也许您还可以补充一点,就是模型必须有点错误,因为否则它将无法泛化,因此无法在其他地方应用。有一些答案说明了这一点。但是现在有太多的答案无法全部阅读。
对我来说,真正的见解在于以下方面:
模型不一定非要正确才有用。
不幸的是,在许多科学中,人们常常忘记了模型不一定必须是现实的精确表示即可允许新的发现和预测!
因此,不要浪费您的时间来构建需要对无数变量进行准确测量的复杂模型。真正的天才发明了可以完成这项工作的简单模型。
package main
import "unsafe"
func f() {
var p uintptr
for i := range [10]int{} {
var x = i
if i == 0 {
// uintptr类型的临时变量只是一个普通的数字,所以其值不应该被改变。
// 因此,x 变量在栈上,10次循环,所使用的栈地址不变,因此 p 最后的值是9
p = uintptr(unsafe.Pointer(&x))
}
}
println(*(*int)(unsafe.Pointer(p))) // 9
}
func g() {
var p unsafe.Pointer
for i := range [10]int{} {
var x = i
if i == 0 {
// 从gc视角来看,unsafe.Pointer是一个指向变量的指针,因此当变量被移动是对应的指针也必须被更新
// 为了跟踪指针更新, x 逃逸到了堆上,10次循环,生成了10个不同的x堆变量, p指向了第1个,其值是0
p = unsafe.Pointer(&x)
}
}
println(*(*int)(p)) // 0
}
func main() {
f()
g()
}
$ go run -gcflags="-m" p.go
# command-line-arguments
./p.go:19:7: moved to heap: x
9
0
从垃圾收集器的视角来看,一个unsafe.Pointer是一个指向变量的指针,因此当变量被移动是对应的指针也必须被更新;但是uintptr类型的临时变量只是一个普通的数字,所以其值不应该被改变。
此例,即考察了unsafe.Pointer
,uintptr
,又考察了变量逃逸。
A先生和S先生,是谁,谷歌一下,原来都是大神啊,大神说的话,应该选择相信。
我摘抄在这里,是想说,我深有同感。我总是从单一的 main.go 开始,随着功能的增加,再分拆结构,也可能一个 main.go 文件就可以了,特别是在制作一些小工具的时候。
Credit: https://unsplash.com/photos/CpkOjOcXdUY
看博客Go is about to get a whole lot faster
源代码:
操作步骤:
cd deque-generic && go1.18beta1 test -run=NONE -bench=. > ../generic.txt
cd deque-non-generic && go1.18beta1 test -run=NONE -bench=. > ../none.txt
cd deque-non-generic && go test -run=NONE -bench=. > ../none-go1.17.6.txt
结果输出:
$ go install golang.org/x/tools/cmd/benchcmp@latest
➜ deque more generic.txt
goos: darwin
goarch: amd64
pkg: github.com/sekoyo/deque
cpu: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
BenchmarkPushFront-12 97286768 12.28 ns/op
BenchmarkPushBack-12 142144150 14.43 ns/op
BenchmarkSerial-12 100000000 11.82 ns/op
BenchmarkSerialReverse-12 100000000 11.21 ns/op
BenchmarkRotate-12 56238 120075 ns/op
BenchmarkInsert-12 36441 120364 ns/op
BenchmarkRemove-12 107545 119688 ns/op
BenchmarkYoyo-12 1777 675390 ns/op
BenchmarkYoyoFixed-12 2662 455608 ns/op
PASS
ok github.com/sekoyo/deque 33.914s
➜ deque more non.txt
goos: darwin
goarch: amd64
pkg: github.com/gammazero/deque
cpu: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
BenchmarkPushFront-12 25286799 50.48 ns/op
BenchmarkPushBack-12 28099020 41.14 ns/op
BenchmarkSerial-12 25631952 50.65 ns/op
BenchmarkSerialReverse-12 26357466 50.56 ns/op
BenchmarkRotate-12 40188 119641 ns/op
BenchmarkInsert-12 28888 120136 ns/op
BenchmarkRemove-12 87488 123670 ns/op
BenchmarkYoyo-12 574 2039762 ns/op
BenchmarkYoyoFixed-12 877 1382135 ns/op
PASS
ok github.com/gammazero/deque 28.086s
➜ deque benchcmp none.txt generic.txt
benchcmp is deprecated in favor of benchstat: https://pkg.go.dev/golang.org/x/perf/cmd/benchstat
benchmark old ns/op new ns/op delta
BenchmarkPushFront-12 50.5 12.3 -75.67%
BenchmarkPushBack-12 41.1 14.4 -64.92%
BenchmarkSerial-12 50.6 11.8 -76.66%
BenchmarkSerialReverse-12 50.6 11.2 -77.83%
BenchmarkRotate-12 119641 120075 +0.36%
BenchmarkInsert-12 120136 120364 +0.19%
BenchmarkRemove-12 123670 119688 -3.22%
BenchmarkYoyo-12 2039762 675390 -66.89%
BenchmarkYoyoFixed-12 1382135 455608 -67.04%
➜ deque more none-go1.17.6.txt
goos: darwin
goarch: amd64
pkg: github.com/gammazero/deque
cpu: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
BenchmarkPushFront-12 24497521 50.09 ns/op
BenchmarkPushBack-12 32226673 39.35 ns/op
BenchmarkSerial-12 28485412 46.10 ns/op
BenchmarkSerialReverse-12 27771453 46.76 ns/op
BenchmarkRotate-12 40466 121028 ns/op
BenchmarkInsert-12 28731 121774 ns/op
BenchmarkRemove-12 86724 125651 ns/op
BenchmarkYoyo-12 620 2049784 ns/op
BenchmarkYoyoFixed-12 820 1389294 ns/op
PASS
ok github.com/gammazero/deque 28.335s
还真是,在 PushFront 上竟然快了 75%,看来等 1.18 正式发布了,还是要香一下的。
go install github.com/nikolaydubina/go-binsize-treemap@latest
go tool nm -size <binary finename> | go-binsize-treemap > binsize.svg
package main
func aaa() (done func(), err error) {
return func() {
print("aaa: done")
}, nil
}
func bbb() (done func(), _ error) {
done, err := aaa()
return func() {
print("bbb: surprise!")
done()
}, err
}
func main() {
done, _ := bbb()
done()
}
这个程序输出结果是什么呢,(单选)?
正确答案是:D,程序最终运行出错。一直调用一直爽,直至栈溢出程序崩溃。
他会不断地递归,疯狂输出 “bbb: surprise!”,直至栈溢出,导致程序运行出错,最终中止。
同学就疑惑了,怎么又多出了个递归?
本质上在函数 bbb 执行完毕后, 变量 done 已经变成了一个递归函数。
递归的过程是:函数 bbb 调用变量 done 后,会输出 bbb: surprise! 字符串,然后又调用变量 done。而变量 done 又是这个闭包(匿名函数),从而实现不断递归调用和输出。
这位大佬出的题目,本质上是比较烦人的,其结合了函数返回参数的命名用法。
如果我们把这个函数的返回参数命名去掉,就可以避开这个问题。
https://mp.weixin.qq.com/s/hcrq5fYaQ7FN_2oSMRNjcA
Go 语言把字符分 byte 和 rune 两种类型处理。byte 是类型 unit8 的别名,用于存放占 1 字节的 ASCII 字符,如英文字符,返回的是字符原始字节。rune 是类型 int32 的别名,用于存放多字节字符,如占 3 字节的中文字符,返回的是字符 Unicode 码点值。如下图所示:
s := "Go语言编程"
// byte
fmt.Println([]byte(s)) // 输出:[71 111 232 175 173 232 168 128 231 188 150 231 168 139]
// rune
fmt.Println([]rune(s)) // 输出:[71 111 35821 35328 32534 31243]
Dockerfile:
FROM golang:alpine AS builder
WORKDIR /build
RUN apk add upx
COPY . .
RUN go build -ldflags "-s -w" -o hello hello.go && upx hello
FROM alpine
WORKDIR /build
COPY --from=builder /build/hello /build/hello
CMD ["./hello"]
docker build -t hello:v1 .
dive hello:v1 .
$ docker run -it --rm hello:v1
hello world!
$ docker run -it --rm hello:v1 ls -lh /build
total 332K
-rwxr-xr-x 1 root root 331.2K Mar 15 02:12 hello
$ docker images | grep hello
hello v1 b3762d8a6c76 12 minutes ago 5.92MB
~/aaa ❯ go install github.com/breml/bidichk/cmd/bidichk@latest
~/aaa 38s ❯ more a.go 14:49:52
package main
import "fmt"
func main() {
str, mask := "Hello, World!<U+202E>10x<U+202D>", 0
bits := 0
for _, ch := range str {
for ch > 0 {
bits += int(ch) & mask
ch = ch >> 1
}
}
fmt.Println("Total bits set:", bits)
}
~/aaa ❯ bidichk a.go 14:49:56
/Users/bingoobjca/aaa/a.go:6:29: found dangerous unicode character sequence RIGHT-TO-LEFT-OVERRIDE
/Users/bingoobjca/aaa/a.go:6:35: found dangerous unicode character sequence LEFT-TO-RIGHT-OVERRIDE
https://goplay.tools/snippet/2Ty5h65xSjv
package main
import "fmt"
func main() {
str := "Hello, World!1234567890"
fmt.Println(str)
}
Output:
Hello, World!1234567890
package main
import "fmt"
func main() {
//str, mask := "Hello, World!10x", 0
str, mask := "Hello, World!<U+202E>10x<U+202D>", 0
bits := 0
for _, ch := range str {
for ch > 0 {
bits += int(ch) & mask
ch = ch >> 1
}
}
fmt.Println("Total bits set:", bits)
str1 := "Hello, World!12345678901 ;"
fmt.Println(str1)
str2 := "Hello, World!12345678901 "
fmt.Println(str2)
}
Total bits set: 0 Hello, World!12345678901 ; Hello, World!12345678901
Program exited.
package main
import (
"fmt"
)
func main() {
v := 4
fmt.Printf("%T\n", v)
f := ((v / 5.0) * 100.0) * 11.0 / 100.0
fmt.Printf("Type of result %T for value %0.2f\n", f, f)
fmt.Printf("\n%T\n", 4)
f2 := ((4 / 5.0) * 100.0) * 11.0 / 100.0
fmt.Printf("Type of result %T for value %0.2f\n", f2, f2)
}
输出结果:
int
Type of result int for value %!f(int=00)
int
Type of result float64 for value 8.80
https://mp.weixin.qq.com/s/bg3LarfaiSKvs5U4mgnEEg
Go Team为Gopher负重前行!
这是在X.com上看到一幅图片,从中可以看出Go team为Gopher的“just write business logic”而负重前行。
就像我在《Go语言精进之路》第一卷中所说的那样:“Go的设计者们在语言设计之初就拒绝走语言特性融合的道路,而选择了“做减法”。并且他们把复杂性留给了语言自身的设计和实现,留给了Go Team自己;而将简单、易用和清晰留给了广大Gopher”。让广大Gopher可以用简单的语法聚焦于编写business logic的代码上。
Golang多版本安装
method1 官方
method2 gvm
install
bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)
influxdb推荐使用gvm来构建: