Golang的一些笔记

[bingoohuang]

Golang多版本安装

method1 官方

$ go get -u golang.org/dl/go1.12.3
go: finding golang.org/dl latest
go: downloading golang.org/dl v0.0.0-20190408222801-b337094d5ff3
go: extracting golang.org/dl v0.0.0-20190408222801-b337094d5ff3
$ go1.12.3 download
Downloaded 100.0% (127615731 / 127615731 bytes)
Unpacking /Users/bingoobjca/sdk/go1.12.3/go1.12.3.darwin-amd64.tar.gz ...
Success. You may now run 'go1.12.3'
$ go1.12.3 version
go version go1.12.3 darwin/amd64
$ go version
go version go1.12.1 darwin/amd64

method2 gvm

install bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)

influxdb推荐使用gvm来构建：

$ gvm install go1.15
$ gvm use go1.15 --default

[bingoohuang]

go继承与覆盖

type HelloWorld interface {
    Hello()
    World()
}

type NullHelloWorld struct{}

func (*NullHelloWorld) Hello() { fmt.Println("Null Hello") }
func (*NullHelloWorld) World() { fmt.Println("Null World") }

type myHello struct {
    NullHelloWorld
}

func (*myHello) Hello() { fmt.Println("my Hello") }

func TestMyHello(t *testing.T) {
    var mh HelloWorld = new(myHello)
    mh.Hello() // my Hello
    mh.World() // Null World
}

[bingoohuang]

Goroutine 泄露

被遗忘的发送者

1

当涉及到内存管理时，Go已经为您处理了许多细节。Go在编译时使用逃逸分析 来决定值在内存中的位置。程序运行时通过使用垃圾回收器 跟踪和管理堆分配。虽然在应用程序中创建内存泄漏 不是不可能的，但是这种可能性已经大大降低了。

一种常见的内存泄漏类型就是 Goroutines 泄漏。如果你开始了一个你认为最终会终止但是它永远不会终止的 Goroutine，那么它就会泄露了。它的生命周期为程序的生命周期，任何分配给 Goroutine 的内存都不能释放。所以在这里建议“永远不要在不知道如何停止的情况下，就去开启一个 Goroutine ”。

示例1

1 // leak 是一个有 bug 程序。它启动了一个 goroutine
2 // 阻塞接收 channel。一切都将不复存在
3 // 向那个 channel 发送数据，并且那个 channel 永远不会关闭
4 // 那个 goroutine 会被永远锁死
5 func leak() {
6      ch := make(chan int)
7 
8      go func() {
9         val := <-ch
10         fmt.Println("We received a value:", val)
11     }()
12 }

示例2

1 // serach 函数得到的返回值用 result 结构体来保存
2 // 通过单个 channel 来传递这两个值
3 type result struct {
4     record string
5     err    error
6 }
7 
8 // process 函数是一个用来寻找记录的函数
9 // 然后打印，如果超过 100 ms 就会失败 .
10 func process(term string) error {
11 
12      // 创建一个在 100 ms 内取消上下文的 context
13      ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
14      defer cancel()
15 
16      // 为 Goroutine 创建一个传递结果的 channel
17      ch := make(chan result)
18 
19      // 启动一个 Goroutine 来寻找记录，然后得到结果
20      // 将返回值从 channel 中返回
21      Go func() {
22          record, err := search(term)
23          ch <- result{record, err}
24      }()
25 
26      // 阻塞等待从 Goroutine 接收值
27      // 通过 channel 和 context 来取消上下文操作
28      select {
29     case <-ctx.Done():
30          return errors.New("search canceled")
31      case result := <-ch:
32          if result.err != nil {
33             return result.err
34          }
35          fmt.Println("Received:", result.record)
36          return nil
37     }
38  }

在第 23 行，它通过channel 发送。在此 channel上发送将阻塞执行，直到另一个 Goroutine准备接收值为止。在超时的情况下，接收方停止等待Goroutine 的接收并继续工作。这将导致 Goroutine 永远阻塞，等待一个永远不会发生的接收器出现。这就是 Goroutine 泄露的时候。

修复：创造一些空间，解决此泄漏的最简单方法是将无缓冲 channel 更改为容量为 1 的缓冲通道。

现在我会给你这个建议：

任何时候你开始 Goroutine 你必须问自己：

1. 什么时候会终止？
2. 什么可以阻止它终止？

并发是一个有用的工具，但必须谨慎使用。

泄漏：被遗弃的接收者,

2

在此内存泄漏示例中，您将看到多个 Goroutines 被阻塞等待接收永远不会发送的值。

文章中程序启动了多个 Goroutines 来处理文件中的一批记录。每个 Goroutine 从输入通道接收值，然后通过输出通道发送新值。

35// processRecords is given a slice of values such as lines
36// from a file. The order of these values is not important
37// so the function can start multiple workers to perform some
38// processing on each record then feed the results back.
39func processRecords(records []string) {
40
41     // Load all of the records into the input channel. It is
42     // buffered with just enough capacity to hold all of the
43     // records so it will not block.
44
45     total := len(records)
46     input := make(chanstring, total)
47     for _, record := range records {
48         input <- record
49     }
50     // close(input) // What if we forget to close the channel?
51
52     // Start a pool of workers to process input and send
53     // results to output. Base the size of the worker pool on
54     // the number of logical CPUs available.
55
56     output := make(chanstring, total)
57     workers := runtime.NumCPU()
58     for i := 0; i < workers; i++ {
59         go worker(i, input, output)
60     }
61
62     // Receive from output the expected number of times. If 10
63     // records went in then 10 will come out.
64
65     for i := 0; i < total; i++ {
66         result := <-output
67         fmt.Printf("[result  ]: output %s\n", result)
68     }
69 }
70
71// worker is the work the program wants to do concurrently.
72// This is a blog post so all the workers do is capitalize a
73// string but imagine they are doing something important.
74//
75// Each goroutine can't know how many records it will get so
76// it must use the range keyword to receive in a loop.
77func worker(id int, input <-chan string, output chan<- string) {
78     for v := range input {
79         fmt.Printf("[worker %d]: input %s\n", id, v)
80         output <- strings.ToUpper(v)
81     }
82     fmt.Printf("[worker %d]: shutting down\n", id)
83 }

该程序从未到达第 82 行，该行将宣布程序正在关闭。即使在 processRecords 函数返回之后，每个 worker Goroutines 仍处于活动状态并等待第 78 行的输入。通道会一直接收数据直到通道关闭并为空。问题是程序永远不会关闭通道。

修复泄漏只需要一行代码: close(input) 。关闭频道是表示”不再发送数据“的一种方式。关闭通道的最合适位置是在第 50 行发送最后一个值之后。

重复这一建议

如果不知道它会如何停止，就不要开始使用 goroutine

[bingoohuang]

在线

package main

import (
    "fmt"
    "time"
)

const (
    ANSIC       = "Mon Jan _2 15:04:05 2006"
    UnixDate    = "Mon Jan _2 15:04:05 MST 2006"
    RubyDate    = "Mon Jan 02 15:04:05 -0700 2006"
    RFC822      = "02 Jan 06 15:04 MST"
    RFC822Z     = "02 Jan 06 15:04 -0700" // RFC822 with numeric zone
    RFC850      = "Monday, 02-Jan-06 15:04:05 MST"
    RFC1123     = "Mon, 02 Jan 2006 15:04:05 MST"
    RFC1123Z    = "Mon, 02 Jan 2006 15:04:05 -0700" // RFC1123 with numeric zone
    RFC3339     = "2006-01-02T15:04:05Z07:00"
    RFC3339Nano = "2006-01-02T15:04:05.999999999Z07:00"
    Kitchen     = "3:04PM"
    // Handy time stamps.
    Stamp      = "Jan _2 15:04:05"
    StampMilli = "Jan _2 15:04:05.000"
    StampMicro = "Jan _2 15:04:05.000000"
    StampNano  = "Jan _2 15:04:05.000000000"
)

func main() {
    fmt.Println(time.Now().Format("2006"))                          // 2020
    fmt.Println(time.Now().Format("2006-01"))                       // 2020-03
    fmt.Println(time.Now().Format("2006-01-02"))                    // 2020-03-03
    fmt.Println(time.Now().Format("2006-01-02 15"))                 // 2020-03-03 04
    fmt.Println(time.Now().Format("2006-01-02 15:04"))              // 2020-03-03 04:41
    fmt.Println(time.Now().Format("2006-01-02 15:04:05"))           // 2020-03-03 04:41:05
    fmt.Println(time.Now().Format("2006-01-02 15:04:05.000"))       // 2020-03-03 04:41:05.721
    fmt.Println(time.Now().Format("2006-01-02 15:04:05.000000"))    // 2020-03-03 04:41:05.721408
    fmt.Println(time.Now().Format("2006-01-02 15:04:05.000000000")) // 2020-03-03 04:41:05.721408664
    fmt.Println(time.Now().Format("2006-01-02 15:04:05Z07:00"))     // 2020-03-03 04:41:05Z
    fmt.Println(time.Now().Local())                                 // 2020-03-03 04:41:05.721408664 +0000 UTC
}

https://stackoverflow.com/a/45163813

Go uses the reference date 20060102150405 which seems meaningless but actually has a reason, as it's 1 2 3 4 5 6 in the Posix date command:

Mon Jan 2 15:04:05 -0700 MST 2006
0   1   2  3  4  5              6

The timezone is 7 but that sits in the middle, so in the end the format resembles 1 2 3 4 5 7 6.

[bingoohuang]

https://play.golang.org/p/K5ZpQsHxlfN

只在我计算机上能跑的代码：select-case-default 忘记让出 CPU 的坑

虽然 select-case-default 会随机均匀的尝试每一个 case，但是并不会主动把 CPU 控制权交出去，需要用runtime.Gosched() 或 <-time.After(time.Microsecond) 把 CPU 让出给其他 goroutine。否则，其他的 goroutine 将可能没有机会运行。

C# 里的 Application.DoEvents() 也是一样的意思，让别的事件有机会被触发。

[bingoohuang]

How to use Diago to diagnose CPU and memory usage in Go programs

Mar 15, 2020 — 3 minutes read Diago is a visualization tool for profiles and heap snapshots generated with pprof.

It is a stand-alone application displaying a call tree of the CPU usage per functions (or per line of codes in functions) but also capable of displaying heap snapshots.

We’ll see in this short article how to use Diago to get more insight in your Go programs and to help you debug them.

Run pprof in the Go program

If you know how to use pprof with Go binaries, this chapter won’t add much to what you already know. If you don’t, you will see that it is quite straight-forward.

There are a few ways of starting profiling using pprof, we will focus on the easiest one: using its HTTP handler.

The Go standard library ships an HTTP handler exposing profiling samples, heap snashots, goroutines information, traces and much more. In order to enable it, you have to import net/http/pprof if you’re already using a HTTP server in your program:

  import _ "net/http/pprof"

If you are not already serving HTTP in your program, the easiest way to get started is to import the packages as mentioned above and to add this snippet to your program:

  go func() {
      log.Println(http.ListenAndServe("localhost:6060", nil))
  }()

With this done, pprof is exposing all the information on http://localhost:6060/debug/pprof.

Get profiles and heap snapshots

Now that pprof is exposing its data through HTTP, it’s quite easy to retrieve profiles and heap snashots.

Profile

A profile will contain information on the amount of time spent by the CPU in each function. It is a proportional value: pprof won’t observe exactly how long each function is running on the CPU, instead, pprof is frequently capturing samples of the CPU usage and they are later aggregated by the visualization tools.

Because the samples are frequently captured by pprof, you can decide how long you want pprof to gather them. Let’s say we want to let pprof run for 30s, we will use curl to download a 30s profile:

curl "http://localhost:6060/debug/pprof/profile?seconds=30" > profile

Heap snapshots

Pprof lets you capture a snapshot of the heap as well. Here’s the command to capture it:

curl "http://localhost:6060/debug/pprof/heap" > heap

We will see in the visualization chapter that there is two kind of information in this file: the total amount of memory allocated per functions since the start of the program and the amount of memory used by each function at the moment of capture.

Visualize the profile and the heap snapshot with Diago

First, if you want to compile and install Diago, use the command:

go get -u github.com/remeh/diago

In order to visualize the profile, start Diago and use the -file flag to provide the filepath of the file to open:

./diago -file profile

and here you go, you should have a visualization such as this one:

You can do the same to open the heap snapshot:

./diago -file heap

In the screenshot above, you can observe that we’re looking at the total memory allocated by each function of the program since it has started (the observed heap in this screenshot is actually a heap snapshot of Diago).

On the top right, you can see a switch to select the other mode displaying the memory usage at the moment of the capture.

Conclusion

There is other tools helping you visualize these files in flamegraph or in a dot diagram, you can even use the pprof command-line tool to display the data by top values and more.

I’ve developed this tool because I sometimes want to visualize this kind of information in a tree and I hope that it could be useful to you as well.

[bingoohuang]

The Go SSA Playground: A web service for exploring Go's SSA intermediate representation. https://github.com/changkun/ssaplayground

[bingoohuang]

Go学习笔记 第三部分 附录

https://wiki.jikexueyuan.com/project/the-go-study-notes-fourth-edition/appendix.html

工具

go build

参数	说明	示例
-gcflags	传递给5g/6g/8g编译器的参数(5:arm, 6:x86-64, 8:x86
-ldflags	传递给5l/6l/8l连接器的参数
-work	查看编译临时目录
-race	允许数据竞争检测（仅支持amd64)
-n	查看但不执行编译命令
-x	查看并执行编译命令
-a	强制重新编译所有依赖包
-v	查看被编译的报名，包括依赖包
-p n	并行编译所使用的CPU core数据。默认全部
-o	输出文件名

gcflags

参数	说明	示例
-B	禁用边界检查
-N	禁用优化
-l	禁用函数内联
-u	禁用unsafe代码
-m	输出优化信息
-S	输出汇编代码

ldflags

参数	说明	示例
-w	禁用DRAWF调试信息，但不包括符号表
-s	禁用符号表
-X	修改字符串符号值	-X main.VER '0.99' -X main.S 'abc'
-H	链接文件类型，其中包括windowsgui	cmd/ld/doc.go

更多参数

1. go tool 6g -h 或 https://golang.org/cmd/gc/
2. go tool 6l -h 或 https://golang.org/cmd/ld/

go install

和 go build 参数相同，将生成文件拷贝到 bin、pkg 目录。优先使用 GOBIN 环境变量所指定目录。

go clean

参数	说明	示例
-n	查看但不执行清理命令
-x	查看并执行清理命令
-i	删除bin pkg目录下的二进制文件
-r	清理所有依赖包临时文件

go get

下载并安装扩展包。默认保存到 GOPATH 指定的第一个工作空间。

参数	说明	示例
-d	仅下载，不执行安装命令
-t	下载测试所需的依赖包
-u	更新包，包括其依赖包
-v	查看并执行命令

go tool objdump

反汇编可执行文件

$ go tool objdump -s "main\.\w+" test
$ go tool objdump -s "main\.main" test

条件编译

通过 runtime.GOOS/GOARCH 判断，或使用编译约束标记。

// +build darwin linux
                        <--- 必须有空行，以区别包文档。
package main

在源文件 (.go, .h, .c, .s 等) 头部添加 "+build" 注释，指示编译器检查相关环境变量。多个约束标记会合并处理。其中空格表示 OR，逗号 AND，感叹号 NOT。

// +build darwin linux --> 合并结果 (darwin OR linux) AND (amd64 AND (NOT cgo))
// +build amd64,!cgo

如果 GOOS、GOARCH 条件不符合，则编译器会会忽略该文件。

还可使用文件名来表示编译约束，比如 test_darwin_amd64.go。使用文件名拆分多个不同平台源文件，更利于维护。

$ ls -l /usr/local/go/src/pkg/runtime

-rw-r--r--@ 1 yuhen admin 11545 11 29 05:38 os_darwin.c
-rw-r--r--@ 1 yuhen admin 1382 11 29 05:38 os_darwin.h
-rw-r--r--@ 1 yuhen admin 6990 11 29 05:38 os_freebsd.c
-rw-r--r--@ 1 yuhen admin 791 11 29 05:38 os_freebsd.h
-rw-r--r--@ 1 yuhen admin 644 11 29 05:38 os_freebsd_arm.c
-rw-r--r--@ 1 yuhen admin 8624 11 29 05:38 os_linux.c
-rw-r--r--@ 1 yuhen admin 1067 11 29 05:38 os_linux.h
-rw-r--r--@ 1 yuhen admin 861 11 29 05:38 os_linux_386.c
-rw-r--r--@ 1 yuhen admin 2418 11 29 05:38 os_linux_arm.c

支持：*_GOOS、*_GOARCH、*_GOOS_GOARCH、*_GOARCH_GOOS 格式。

可忽略某个文件，或指定编译器版本号。更多信息参考标准库 go/build 文档。

// +build ignore
// +build go1.2 <--- 最低需要 go 1.2 编译。

自定义约束条件，需使用 "go build -tags" 参数。

test.go

// +build beta,debug

package main

func init() {
    println("test.go init")
}

输出：

$ go build -tags "debug beta" && ./test
test.go init

$ go build -tags "debug" && ./test
$ go build -tags "debug \!cgo" && ./test

考虑到这一点，我这样扩展了 Peter 的建议：

// +build unit - 不需要任何服务的测试,
// +build integration - 一个强制标记来测试我们自己的服务，
// +build external - 针对第三方和公共服务进行测试，
// +build integration,external - 针对我们自己的服务以及公共服务进行测试,
// +build !integration,external - 专门针对公共服务进行测试,
// +build unit integration - 不依赖服务，提供集成功能

我们的测试通常属于单元测试、集成测试或外部测试的某一类，或者是它们的某种组合。我们肯定希望在 CI 任务中跳过 external 测试，原因显而易见，但如果我们正在考虑调试开发中的一些相关问题，它们是非常有价值的。我们经常需要定位到具体包中具体的测试，因此运行类似下面的内容是有意义的：

go test --tags="integration external" ./messaging/webhooks/...

跨平台编译

首先得生成与平台相关的工具和标准库。

$ cd /usr/local/go/src

$ GOOS=linux GOARCH=amd64 ./make.bash --no-clean

# Building C bootstrap tool.
cmd/dist

# Building compilers and Go bootstrap tool for host, darwin/amd64.

cmd/6l
cmd/6a
cmd/6c
cmd/6g
...
---
Installed Go for linux/amd64 in /usr/local/go
Installed commands in /usr/local/go/bin

说明：参数 no-clean 避免清除其他平台文件。 然后回到项目所在目录，设定 GOOS、GOARCH 环境变量即可编译目标平台文件。

$ GOOS=linux GOARCH=amd64 go build -o test

$ file test
learn: ELF 64-bit LSB executable, x86-64, version 1 (SYSV)

$ uname -a
Darwin Kernel Version 12.5.0: RELEASE_X86_64 x86_64

预处理

简单点说，go generate 扫描源代码文件，找出所有 "//go:generate" 注释，提取并执行预处理命令。

• 命令必须放在 .go 文件。
• 每个文件里可以有多个 generate 指令。
• 必须显式用 go generate 执行。
• 命令行支持环境变量。
• 按文件名顺序依次提取执行。
• 串行执行，出错终止。
• 必须以 "//go:generate" 开头，双斜线后没有空格。

不属于 build 组成部分，设计目标是提供给包开发者使用，因为包用户可能不具备命令执行环境。

//go:generate ls -l
//go:generate du

参数	说明	示例
-x	显示并执行命令
-n	显示但不执行命令
-v	输出处理的包和源文件名

还可定义别名。须提前定义，仅在当前文件内有效。

//go:generate -command YACC go tool yacc
//go:generate YACC -o test.go -p parse test.y

可用条件编译，让 go build 忽略包含 generate 的文件。

// +build generate

$ go generate -tags generate

资源：

1. Design Document
2. Generating code

调试

1. GDB

默认情况下，编译的二进制文件已包含 DWARFv3 调试信息，只要 GDB 7.1 以上版本都可以调试。

相关选项：

• 调试: 禁用内联和优化 -gcflags "-N -l"。
• 发布: 删除调试信息和符号表 -ldflags "-w -s"。

除了使用 GDB 的断点命令外，还可以使用 runtime.Breakpoint 函数触发中断。另外，runtime/debug.PrintStack 可用来输出调用堆栈信息。

某些时候，需要手工载入 Go Runtime support (runtime-gdb.py)。

.gdbinit

define goruntime
    source /usr/local/go/src/runtime/runtime-gdb.py
end

set disassembly-flavor intel
set print pretty on
dir /usr/local/go/src/pkg/runtime

2. Data Race

数据竞争 (data race) 是并发程序里不太容易发现的错误，且很难捕获和恢复错误现场。Go 运行时内置了竞争检测，允许我们使用编译器参数打开这个功能。它会记录和监测运行时内存访问状态，发出非同步访问警告信息。

func main() {
    var wg sync.WaitGroup
    wg.Add(2)
    x := 100

    go func() {
        defer wg.Done()

        for {
            x += 1
        }
    }()

    go func() {
        defer wg.Done()
        for {
            x += 100
        }
    }()

    wg.Wait()
}

输出：

$ GOMAXPROCS=2 go run -race main.go

==================
WARNING: DATA RACE
Write by goroutine 4:
    main.func·002()
        main.go:25 +0x59

Previous write by goroutine 3:
    main.func·001()
        main.go:18 +0x59

Goroutine 4 (running) created at:
    main.main()
        main.go:27 +0x16f
Goroutine 3 (running) created at:
    main.main()
        main.go:20 +0x100
==================

数据竞争检测会严重影响性能，不建议在生产环境中使用。

func main() {
    x := 100

    for i := 0; i < 10000; i++ {
        x += 1
    }

    fmt.Println(x)
}

输出：

$ go build && time ./test

10100

real" 0m0.060s
user" 0m0.001s
sys" 0m0.003s

$ go build -race && time ./test

10100

real" 0m1.025s
user" 0m0.003s
sys" 0m0.009s

通常作为非性能测试项启用。

$ go test -race

测试

自带代码测试、性能测试、覆盖率测试框架。

• 测试代码必须保存在 *_test.go 文件。
• 测试函数命名符合 TestName 格式，Name 以大写字母开头。

注: 不要将代码放在名为 main 的目录下，这会导致 go test "cannot import main" 错误。

1. Test 使用 testing.T 相关方法决定测试状态。

testing.T

方法|说明|其它 Fail|标记失败，但继续执行该测试函数| FailNow|失败，立即停止当前测试函数| Log|输出信息。尽在失败或者-v参数时输出|Logf SkipNow|跳过当前测试函数 | Skipf = SkipNow + Logf Error | Fail + Log | Errorf Fatal | FailNow + Log | Fatalf

main_test.go

package main

import (
    "testing"
    "time"
)

func sum(n ...int) int {
    var c int
    for _, i := range n {
        c += i
    }

    return c
}

func TestSum(t *testing.T) {
    time.Sleep(time.Second * 2)
    if sum(1, 2, 3) != 6 {
        t.Fatal("sum error!")
    }
}

func TestTimeout(t *testing.T) {
    time.Sleep(time.Second * 5)
}

默认 go test 执行所有单元测试函数，支持 go build 参数。

参数	说明	示例
-c	仅编译，不执行测试
-v	显示所有测试函数执行细节
-run regex	执行指定的测试函数（正则表达式)
-parallel n	并发执行测试函数（默认: GOMAXPROCS）
-timeout t	单个测试超时时间	-timeout 2m30s

$ go test -v -timeout 3s

=== RUN TestSum
--- PASS: TestSum (2.00 seconds)
=== RUN TestTimeout
panic: test timed out after 3s
FAIL" test" 3.044s

$ go test -v -run "(i)sum"

=== RUN TestSum
--- PASS: TestSum (2.00 seconds)
PASS
ok " test" 2.044s

可重写 TestMain 函数，处理一些 setup/teardown 操作。

func TestMain(m *testing.M) {
    println("setup")
    code := m.Run()
    println("teardown")
    os.Exit(code)
}

func TestA(t *testing.T) {}
func TestB(t *testing.T) {}
func BenchmarkC(b *testing.B) {}

输出：

$ go test -v -test.bench .

setup
=== RUN TestA
--- PASS: TestA (0.00s)
=== RUN TestB
--- PASS: TestB (0.00s)
PASS
BenchmarkC" 2000000000" 0.00 ns/op
teardown
ok " test" 0.028s

2. Benchmark

性能测试需要运行足够多的次数才能计算单次执行平均时间。

func BenchmarkSum(b *testing.B) {
    for i := 0; i < b.N; i++ {
        if sum(1, 2, 3) != 6 {
            b.Fatal("sum")
        }
    }
}

默认情况下，go test 不会执行性能测试函数，须使用 "-bench" 参数。

go test

参数	说明	示例
-bench regex	执行指定性能测试函数（正则表达式）
-benchmem	输出内存统计信息
-benchtime -t	设置每个性能测试运行时间	-benchtime 1m30s
-cpu	设置并发测试。 默认 GOMAXPROCS	-cpu 1,2,4

$ go test -v -bench .

=== RUN TestSum
--- PASS: TestSum (2.00 seconds)
=== RUN TestTimeout
--- PASS: TestTimeout (5.00 seconds)
PASS

BenchmarkSum 100000000 11.0 ns/op

ok " test" 8.358s

$ go test -bench . -benchmem -cpu 1,2,4 -benchtime 30s

BenchmarkSum 5000000000 11.1 ns/op 0 B/op 0 allocs/op
BenchmarkSum-2 5000000000 11.4 ns/op 0 B/op 0 allocs/op
BenchmarkSum-4 5000000000 11.3 ns/op 0 B/op 0 allocs/op

ok " test" 193.246s

Example

与 testing.T 类似，区别在于通过捕获 stdout 输出来判断测试结果。

func ExampleSum() {
    fmt.Println(sum(1, 2, 3))
    fmt.Println(sum(10, 20, 30))
    // Output:
    // 6
    // 60
}

不能使用内置函数 print/println，它们默认输出到 stderr。

$ go test -v

=== RUN: ExampleSum
--- PASS: ExampleSum (8.058us)
PASS

ok " test" 0.271s

Example 代码可输出到文档，详情参考包文档章节。

Cover

除显示代码覆盖率百分比外，还可输出详细分析记录文件。

参数	说明
-cover	允许覆盖分析
-covermode	代码分析模式（set: 是否执行；count:执行次数； atomic: 次数，并发支持)
-coverprofile	输出结果文件

$ go test -cover -coverprofile=cover.out -covermode=count

PASS
coverage: 80.0% of statements
ok " test" 0.043s

$ go tool cover -func=cover.out

test.go: Sum 100.0%
test.go: Add 0.0%
total:" (statements) 80.0%

用浏览器输出结果，能查看更详细直观的信息。包括用不同颜色标记覆盖、运行次数等。

$ go tool cover -html=cover.out

说明：将鼠标移到代码块，可以看到具体的执行次数。

PProf

监控程序执行，找出性能瓶颈。

import (
    "os"
    "runtime/pprof"
)

func main() {
    // CPU
    cpu, _ := os.Create("cpu.out")
    defer cpu.Close()
    pprof.StartCPUProfile(cpu)
    defer pprof.StopCPUProfile()

    // Memory
    mem, _ := os.Create("mem.out")
    defer mem.Close()
    defer pprof.WriteHeapProfile(mem)
}

除调用 runtime/pprof 相关函数外，还可直接用测试命令输出所需记录文件。

参数|说明 -blockprofile block.out|goroutine阻塞 -blockprofilerate n | 超出该参数设置时间 的阻塞才被记录，单位：纳秒 -cpuprofile cpu.out | CPU -memprofile mem.out | 内存分配 -memprofilerate n | 超出该参数(bytes)设置的内存分配才被记录，默认 512KB (mprof.go)

以 net/http 包为演示，先生成记录文件。

$ go test -v -test.bench "." -cpuprofile cpu.out -memprofile mem.out net/http

进入交互式查看模式。

$ go tool pprof http.test mem.out

(pprof) top5
2597.58kB of 2597.58kB total ( 100%)
Dropped 421 nodes (cum <= 12.99kB)
Showing top 5 nodes out of 28 (cum >= 1536.60kB)

     flat  flat%    sum%       cum     cum%
1024.04kB 39.42%  39.42% 1024.04kB   39.42% encoding/asn1.parsePrintableString
 548.84kB 21.13%  60.55%  548.84kB   21.13% mime.setExtensionType
 512.56kB 19.73%  80.28% 1536.60kB   59.16% crypto/x509.parseCertificate
 512.14kB 19.72%    100%  512.14kB   19.72% mcommoninit
      0        0%   100% 1536.60kB   59.16% crypto/tls.(*Conn).Handshake

• flat: 仅当前函数，不包括其调用的其他函数。
• sum: 列表前几行所占百分比总和。
• cum: 当前函数完整调用堆栈。

默认输出 inuse_space，可在命令行指定其他值，包括排序方式。

$ go tool pprof -alloc_space -cum http.test mem.out

可输出函数调用的列表统计信息。

或者是更详细的源码模式。

除交互模式外，还可直接输出统计结果。

$ go tool pprof -text http.test mem.out

2597.58kB of 2597.58kB total ( 100%)
Dropped 421 nodes (cum <= 12.99kB)

     flat  flat%   sum%       cum   cum%
1024.04kB 39.42% 39.42% 1024.04kB 39.42% encoding/asn1.parsePrintableString
 548.84kB 21.13% 60.55%  548.84kB 21.13% mime.setExtensionType
 512.56kB 19.73% 80.28% 1536.60kB 59.16% crypto/x509.parseCertificate
 512.14kB 19.72%   100%  512.14kB 19.72% mcommoninit
      0      0%    100% 1536.60kB 59.16% crypto/tls.(*Conn).Handshake
      0      0%    100% 1536.60kB 59.16% crypto/tls.(*Conn).clientHandshake

输出图形文件。

$ go tool pprof -web http.test mem.out

还可用 net/http/pprof 实时查看 runtime profiling 信息。

package main

import (
    _ "net/http/pprof"

    "net/http"
    "time"
)

func main() {
    go http.ListenAndServe("localhost:6060", nil)

    for {
        time.Sleep(time.Second)
    }
}

在浏览器中查看 http://localhost:6060/debug/pprof/ 。

附: 自定义统计数据，可用 expvar 导出，用浏览器访问 /debug/vars 查看。

[bingoohuang]

Find out the version of Go a Binary was built with?

➜  gonginx git:(master) go install ./...                                    [一  6/15 15:13:46 2020]
➜  gonginx git:(master) strings ~/go/bin/gonginx | grep 'Go build\|go1\.'   [一  6/15 15:13:49 2020]
stack=[address bad instboundarycgocheckcontinuedefault:durationencodingexporterfc00::/7filenamefinishedfont/otffont/ttfgo1.14.2hijackedhttp/1.1https://if-matchif-rangeinfinityinvalid locationloopbackno-cacheno_proxyopPseudooverflowraw-readreadfromrecvfromreflect.runnableruntime.scavengeshutdownstrconv.templatethetasymtimeout:truncateunixgramunknown(urlquery (forced) -> node= as type  blocked= defersc= in use)
/usr/local/go/src/vendor/golang.org/x/crypto/poly1305/bits_go1.13.go
 Go buildinf:
➜  gonginx git:(master) go install -ldflags="-s -w" ./...                  [一  6/15 15:13:50 2020]
➜  gonginx git:(master) strings ~/go/bin/gonginx | grep 'Go build\|go1\.'  [一  6/15 15:13:55 2020]
stack=[address bad instboundarycgocheckcontinuedefault:durationencodingexporterfc00::/7filenamefinishedfont/otffont/ttfgo1.14.2hijackedhttp/1.1https://if-matchif-rangeinfinityinvalid locationloopbackno-cacheno_proxyopPseudooverflowraw-readreadfromrecvfromreflect.runnableruntime.scavengeshutdownstrconv.templatethetasymtimeout:truncateunixgramunknown(urlquery (forced) -> node= as type  blocked= defersc= in use)
/usr/local/go/src/vendor/golang.org/x/crypto/poly1305/bits_go1.13.go
 Go buildinf:
➜  gonginx git:(master) go install ./... && upx --best --lzma ~/go/bin/gonginx   [一  6/15 15:14:15 2020]
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2018
UPX 3.95        Markus Oberhumer, Laszlo Molnar & John Reiser   Aug 26th 2018

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
  11508820 ->   5824528   50.61%   macho/amd64   gonginx

Packed 1 file.
➜  gonginx git:(master) strings ~/go/bin/gonginx | grep 'Go build\|go1\.'  [一  6/15 15:14:26 2020]
 Go buildinf:
➜  gonginx git:(master) go install -ldflags="-s -w" ./... && upx ~/go/bin/gonginx  [一  6/15 15:14:28 2020]
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2018
UPX 3.95        Markus Oberhumer, Laszlo Molnar & John Reiser   Aug 26th 2018

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
   9182292 ->   3493904   38.05%   macho/amd64   gonginx

Packed 1 file.
➜  gonginx git:(master) strings ~/go/bin/gonginx | grep 'Go build\|go1\.'        [一  6/15 15:14:44 2020]
 Go build ID: "MokNR
 Go buildinf:
➜  gonginx git:(master) strings ~/go/bin/gonginx | grep 'UPX\|Go build\|go1\.'   [一  6/15 15:23:12 2020]
UPX!
 Go build ID: "MokNR
 Go buildinf:
UPX!
$Info: This file is packed with the UPX executable packer http://upx.sf.net $
$Id: UPX 3.95 Copyright (C) 1996-2018 the UPX Team. All Rights Reserved. $
UPX!t
UPX!

[bingoohuang]

https://go-proverbs.github.io/

Go箴言

[bingoohuang]

结构体能不能判断等于/不等于

from blog Go with Examples: Keeping Your Modules Compatible

Struct values are comparable if all their fields are comparable. Two struct values are equal if their corresponding non-blank fields are equal.

• Golang Reference Specification

In other words, if you add on a slice, map or function field into a struct, we will not be able to perform a comparison on the struct.

https://play.golang.org/p/j533RPDkOSo

package main

import (
    "fmt"
)

// OK!
type ComparableStruct struct {
   i int
   s string
   b bool
}

// Unable to perform == or != operation on this struct
type UncomparableStruct struct {
   i int
   s string
   b bool
   v []int
}

func main() {
   comparable1 := ComparableStruct{1,"s", true}
   comparable2 := ComparableStruct{1,"s", true}
   uncomparable1 := UncomparableStruct{1,"s", true, []int{1,2,3}}
   uncomparable2 := UncomparableStruct{1,"s", true, []int{1,2,3}}
   fmt.Println(comparable1 == comparable2)
   fmt.Println(uncomparable1 == uncomparable2) //error not comparable
   // ./prog.go:28:30: invalid operation: uncomparable1 == uncomparable2 (struct containing []int cannot be compared)
}

[bingoohuang]

跨包接口的兼容性

一篇博客的例子

When an Interface Depends on Another Interface in Go

textworker/textworker.go

package textworker

import "fmt"

type Worker struct {}

func (w Worker) Do() {
  fmt.Println("I am working")
}

async/asyncrunner.go

package async

type Worker interface {
   Do()
}

type Runner struct{}

func (r Runner) Run(w Worker) {
   go w.Do()
}

main.go

package main

import (
   "async"
   "runner"
   "textworker"
)

func main() {  
   worker := textworker.Worker{}
   asyncRunner := async.Runner{}

   runner.Run(asyncRunner, worker) // compiler error
}

🕙[2020-09-02 09:47:42.838] ❯ go build                
# interfacedep
./main.go:13:12: cannot use asyncRunner (type async.Runner) as type runner.Runner in argument to runner.Run:
        async.Runner does not implement runner.Runner (wrong type for Run method)
                have Run(async.Worker)
                want Run(runner.Worker)

改进方法1：

type Runner interface {
  Run(workerFunc func())
}

改进方法2:

type Runner interface {
  Run(w interface{
    Do()
  })
}

改进方法2.1，匿名接口内嵌Worker

type Runner interface {
  Run(w interface{Worker})
}

更多的例子

Example of cross-package interfaces in golang上的例子

test_a_sayer.go

package a

import "fmt"

type Sayer interface {
    Say() string
}

type Formal struct{}

func (p Formal) Greet(s interface{Sayer}) string {
    return fmt.Sprintf("Hello, %s", s.Say())
}

test_b_sayer.go:

package b

import "fmt"

type Sayer interface {
    Say() string
}

type Casual struct{}

func (p Casual) Greet(s interface{Sayer}) string {
    return fmt.Sprintf("Hey %s!", s.Say())
}

test_greet_greet.go:

package greet

type Sayer interface {
    Say() string
}

type Greeter interface {
    Greet(s interface{Sayer}) string
}

func Greet(g interface{Greeter}, i Sayer) string {
    return g.Greet(i)
}

test_main.go:

package main

import (
    "log"
    "test/a"
    "test/b"
    "test/greet"
)

type Who struct {
    to string
}

func (w Who) Say() string {
    return w.to
}

type Bad struct {
    to string
}

func (w Bad) Say(exclaim bool) string {
    said := w.to
    if exclaim {
        said += "!!!"
    }
    return said
}

func main() {
    log.Println("Testing...")

    a := a.Formal{}
    b := b.Casual{}

    w := &Who{
        to:"world",
    }

    x := &Bad{
        to:"world",
    }

    log.Println(greet.Greet(a, w))
    log.Println(greet.Greet(b, w))
    //log.Println(greet.Greet(b, x))
}

[bingoohuang]

并发bug发现不易的一个例子

FINDING GOROUTINE BUGS WITH TLA+

https://github.com/google/gops/blob/5d514cabbb21de80cb27f529555ee090b60bbb80/goprocess/gp.go#L29

/*1 */ func FindAll() []P { //P = process data
/*2 */    pss, err := ps.Processes()
/*3 */    [...]
/*4 */    found := make(chan P)
/*5 */    limitCh := make(chan struct{}, concurrencyProcesses)
/*6 */ 
/*7 */    for _, pr := range pss {
/*8 */       limitCh <- struct{}{}
/*9 */       pr := pr
/*10*/       go func() {
/*11*/          defer func() { <-limitCh }()
/*12*/          [... get a P with some error checking ...]
/*13*/          found <- P
/*14*/       }()
/*15*/    }
/*16*/    [...]
/*17*/ 
/*18*/    var results []P
/*19*/    for p := range found {
/*20*/       results = append(results, p)
/*21*/    }
/*22*/    return results
/*23*/ }

以上代码，可能会出现死锁。其实有两个并发的bug。

Even in Go, concurrency is still not easy (with an example) September 1, 2020 Go is famous for making concurrency easy, through good language support for goroutines. Except what Go makes easy is only one level of concurrency, the nuts and bolts level of making your code do things concurrently and communicating back and forth through channels. Making it do the right things concurrently is still up to you, and unfortunately Go doesn't currently provide a lot of standard library support for correctly implemented standard concurrency patterns.

For example, one common need is for a limited amount of concurrency; you want to do several things at once, but only so many of them. At the moment this is up to you to implement on top of goroutines, channels, and things like the sync package. This is not as easy as it looks, and quite competent people can make mistakes here. As it happens, I have an example ready to hand today.

Gops is a convenient command to list (and diagnose) Go processes that are currently running on your system. Among other things, it'll tell you which version of Go they were compiled with, which is handy if you want to see if you have out of date binaries that should be rebuilt and redeployed. One of the things gops needs to do is look at all of the Go processes on your system, which it does concurrently. However, it doesn't want to look at too many processes at once, because that can cause problems with file descriptor limits. This is a classic case of limited concurrency.

Gops implements this at the moment with code in goprocess.FindAll() that looks like this, in somewhat sketched and reduced form:

func FindAll() []P { pss, err := ps.Processes() [...] found := make(chan P) limitCh := make(chan struct{}, concurrencyProcesses)

for _, pr := range pss { limitCh <- struct{}{} pr := pr go func() { defer func() { <-limitCh }() [... get a P with some error checking ...] found <- P }() } [...]

var results []P for p := range found { results = append(results, p) } return results } (In the real code there's a WaitGroup for coordination, and the found channel gets closed appropriately.)

How this works is clear, and is a standard pattern (covered in eg Go 101's Channel Use Cases). We use a buffered channel to provide a limited number of tokens; sending a value into the channel implicitly takes a token (and blocks if the token supply is exhausted), while receiving a value from it puts a token back in. We take a token before we start a new goroutine, and the goroutine releases the token when it's done.

Except that this code has a bug if there are too many processes to examine. Even knowing that there is a bug in this code, it may not be obvious.

The bug is that the goroutines only receive from limitCh to release their token after sending their result to the unbuffered found channel, while the main code only starts receiving from found after running through the entire loop, and the main code takes the token in the loop and blocks if no tokens are available. So if you have too many processes to go through, you start N goroutines, they all block trying to write to found and don't receive from limitCh, and the main for loop blocks trying to send to limitCh and never reaches the point where it starts receiving from found.

[bingoohuang]

三款 Go Playground

对比三款 Go Playground：你喜欢哪款？

1. Better Go Playground goplay.tools
2. goplay.space
3. 官方的 Playground，国内版

[bingoohuang]

并行与并发的图解

What is Concurrency? Is it the same as Threaded? Parallel? Asynchronous?

Concurrency in its simplest form can be defined as doing multiple things at once. Imagine writing two letters with both hands when you have one pencil, you write a sentence with your right hand on one letter, then write a sentence with your left hand on other letter. You might keep alternating hands, until both the letters are complete. — Doing multiple tasks, by sharing time, resources etc.

Parallelism means starting the tasks, completing the tasks together. Imagine writing the letters with both hands with a pencil in each hand without interruption. This would require a lot of brain power and practice. — Doing multiple tasks “at the same time”

原文：Go: A Tale of Concurrency ( A Beginners Guide )

[bingoohuang]

我的看法：明天将要发布的代码选择 Go，在未来五年内保持不变的代码选择 Rust。—Grzegorz Nosek

如果你想加快开发速度，也许是因为你要编写许多不同的服务，或者你有庞大的开发团队，那么 Go 是你选择的语言。Go 为你提供了一流的并发性，并且不容许不安全的内存访问（Rust 也不容忍），但不会强迫你管理每个最后的细节。Go 是快速而强大的工具，但是它避免了使开发人员陷入困境，而专注于简单性和统一性。另一方面，如果需要拧紧块性能，那么 Rust 应该是你的选择。—Andrew Lader[23]

来自也许是最客观、全面的比较 Rust 与 Go：都想把 Rust 也学一下，原文链接

[bingoohuang]

Find your leaf packages loov/goda:GoDependencyAnalysistoolkit

1. go get github.com/loov/goda
2. goda graph ./... | dot -Tsvg -o graph.svg

[bingoohuang]

Golang枚举生成工具

enumer，是stringer的fork上增强的版本

1. go get github.com/alvaroloes/enumer
2. enumer -type=Pill -json -transform=snake
3. stringer的用法参考, only generates String() methods, leaving MarshalText() and UnmarshalText() unimplemented.

[bingoohuang]

有一种未经证实的说法：

Go诞生于C++程序的漫长构建过程中。

如果C++编译很快，那么Robert Griesemer、Rob Pike和Ken Thompson这三位大佬也没有闲暇时间一起喝着咖啡并决定是时候设计一门新语言了。的确，Go语言诞生后，其简洁的语法、极速地构建、新颖的并发结构、体验优良的工具链以及完成度不低的标准库吸引了很多C/C++程序员转型成为Gopher并开始重度使用Go

来自重度使用 Go 的“后遗症“，你有吗？

1. https://commandcenter.blogspot.com/2012/06/less-is-exponentially-more.html
2. https://mikespook.com/2012/06/翻译少是指数级的多/

回到 2007 年 9 月，我在一个巨大的 Google C++ 程序（就是你们都用过的那个）上做一些琐碎但是很核心的工作，我在那个巨大的分布式集群上需要花大约 45 分钟进行编译。收到一个通知说 Google 雇佣的一对为 C++ 标准化委员会工作的夫妇将会做一场报告。收到一个通知说几个受雇于 Google 的为 C++ 标准化委员会工作的人将会做一场报告。他们将向我们介绍那时还被称作 C++0x（就是现在众所周知的 C++11）中将会有哪些改进。

在长达一个小时的报告中，我们听说了诸如有已经在计划中的 35 个特性之类的事情。事实上有更多，但仅有 35 个特性在报告中进行了描述。当然一些特性很小，但是意义重大，值得在报告中提出。一些非常微妙和难以理解，如左右值引用（rvalue references），还有一些是 C++ 特有的，如可变参数模板（variadic templates），还有一些就是发疯，如用户定义数据标识（user-defined literals）。

这时我问了自己一个问题：C++ 委员会真得相信 C++ 的问题在于没有足够的特性？肯定的说，在另一个 Ron Hardin 的玩笑中，简化语言的成就远远大于添加功能。当然这有点可笑，不过请务必记住这个思路。

[bingoohuang]

Different Thread Models

There are three different threading models we can see Mx1, 1x1, MxN

Go language implemented MxN with three basic primitive entities

• G Structure — A G struct represents a single goroutine, stack, current stack, stack gaurd, pointer to code (initial function), parameters, goID
• P Structure — Abstraction to the processor in Go runtime. It holds the Context to run Go routine.
• M Structure — The M struct is the Go runtime’s representation of an OS thread, global queue of G’s, the G that it is currently running, its own cache, and a handle to the scheduler.

FROM Different Threading Models — Why I Feel Go Threading Is Better

[bingoohuang]

Go Slice Tricks Cheat Sheet

[bingoohuang]

应该有很多人受不了err的反复判断，封装各种err处理，达到简化的目的，其中一种实现ErrorFlow Declarative error handling for Go.

func GzipFile(dstFilename string, srcFilename string) (err error) {
    // defer IfError()... creates and configures
    // ErrorFlow error handler for this function.
    // When any of Check* functions encounters non-nil error
    // it immediately sends error to this handler
    // unwinding all stacked defers.
    errWrapper := errf.WrapperFmtErrorw("error compressing file")
    defer errf.IfError().ReturnFirst().LogIfSuppressed().Apply(errWrapper).ThenAssignTo(&err)

    errf.CheckCondition(len(dstFilename) == 0, "dst file should be specified")
    errf.CheckCondition(len(srcFilename) == 0, "src file should be specified")

    reader := errf.Io.CheckReadCloser(os.Open(srcFilename))
    defer errf.With(errWrapper).Log(reader.Close())

    writer := errf.Io.CheckWriteCloser(os.Create(dstFilename))
    defer errf.Handle().OnAnyErrOrPanic(func() { os.Remove(dstFilename) })
    defer errf.CheckErr(writer.Close())

    gzipWriter := gzip.NewWriter(writer)
    defer errf.CheckErr(gzipWriter.Close())

    return errf.CheckDiscard(io.Copy(gzipWriter, reader))
}

[bingoohuang]

vendor 打包编译

开发机上

$ cd app
$ go mod download -v
$ go mod vendor
$ cd ..
$ tar --exclude .git --exclude .idea -czf app.tar.gz app

编译机上

$ tar zxf app.tar.gz
$ cd app
$ go build -mod vendor -o app   -ldflags=' -w -s '

[bingoohuang]

给expvarmon插上数据持久化的“翅膀”

package main

import (
    _ "expvar"
    "net/http"
)

func main() {
    http.ListenAndServe(":8100", nil)
}

1. go install github.com/divan/expvarmon@latest
2. expvarmon -ports="8100"

[bingoohuang]

Serve embedded filesystem from root path of URL

package main

import (
    "embed"
    "io/fs"
    "log"
    "net/http"
)

//go:embed static
var embeddedFS embed.FS

func main() {
    serverRoot, err := fs.Sub(embeddedFS, "static")
    if err != nil {
        log.Fatal(err)
    }

    http.Handle("/", http.FileServer(http.FS(serverRoot)))
    log.Fatal(http.ListenAndServe(":8080", nil))
}

[bingoohuang]

go可执行文件分析工具redress

The redress software is a tool for analyzing stripped Go binaries compiled with the Go compiler. It extracts data from the binary and uses it to reconstruct symbols and performs analysis. It essentially tries to "re-dress" a "stripped" binary.

🕙[2021-05-05 22:16:16.837] ❯ redress -help
Usage of redress:
  -compiler
        Print information
  -filepath
        Include file path for packages
  -force-version string
        Forcing and using the given version when analyzing
  -interface
        Print interfaces
  -method
        Print type's methods
  -pkg
        List packages
  -src
        Print source tree
  -std
        Include standard library packages
  -struct
        Print structs
  -type
        Print all type information
  -unknown
        Include unknown packages
  -vendor
        Include vendor packages
  -version
        Print redress version

🕙[2021-05-05 22:15:51.625] ❯ go install -ldflags="-s -w" ./...

🕙[2021-05-05 22:16:01.530] ❯ ls -lh ~/go/bin/shorturl
-rwxr-xr-x  1 bingoo  staff    15M  5  5 22:16 /Users/bingoo/go/bin/shorturl

🕙[2021-05-05 22:15:22.711] ❯ redress -src ~/go/bin/shorturl
Package main: /Users/bingoo/GitHub/shorturl
File: main.go
    main Lines: 12 to 22 (10)
Package github.com/bingoohuang/shorturl/pkg: /Users/bingoo/GitHub/shorturl/pkg
File: <autogenerated>
    (*noCache)DeactivateURL Lines: 1 to 1 (0)
    (*noCache)SavePopularURL Lines: 1 to 1 (0)
    (*RedisCache)DeactivateURL Lines: 1 to 1 (0)
    (*noCache)LookupURL Lines: 1 to 1 (0)
    (*RedisCache)SavePopularURL Lines: 1 to 1 (0)
    (*Body)Merge Lines: 1 to 1 (0)
    (*URLInput)ValidateExpiry Lines: 1 to 1 (0)
    (*RedisCache)LookupURL Lines: 1 to 1 (0)
    (*URL)IsActive Lines: 1 to 1 (0)
    (*URLInput)GetExpiresOn Lines: 1 to 1 (0)
File: admin.go
    ListURLsFilteredFromRequest Lines: 9 to 21 (12)
    ListURLsFiltered Lines: 21 to 46 (25)
    DeleteURLFromRequest Lines: 46 to 53 (7)
    DeleteURLByShortCode Lines: 53 to 65 (12)
File: client.go
    CreateURLShortCodeFromRequest Lines: 13 to 25 (12)
    CreateURLShortCode Lines: 25 to 55 (30)
    LookupOriginURL Lines: 55 to 76 (21)
    IncrementHits Lines: 76 to 87 (11)
    ValidateURLInput Lines: 87 to 115 (28)
    getUniqueShortCode Lines: 115 to 133 (18)
    isShortCodeAvail Lines: 133 to 143 (10)
    getShortCodeByOriginURL Lines: 143 to 153 (10)
    mapKeywords Lines: 153 to 162 (9)
File: config.go
    init Lines: 28 to 28 (0)
    createDB Lines: 49 to 74 (25)
    init0 Lines: 74 to 90 (16)
    init.0func1 Lines: 75 to 79 (4)
    createEnvName Lines: 90 to 100 (10)
File: controller.go
    ListURLs Lines: 10 to 22 (12)
    DeleteShortURL Lines: 22 to 33 (11)
    CreateShortURL Lines: 33 to 47 (14)
    NotFound Lines: 47 to 53 (6)
    ServeShortURL Lines: 53 to 68 (15)
File: model.go
    URLIsActive Lines: 55 to 100 (45)
    (*URLInput)Validate Lines: 100 to 142 (42)
    URLInputValidateExpiry Lines: 142 to 164 (22)
    URLInputGetExpiresOn Lines: 164 to 173 (9)
    URLFilterGetOffset Lines: 173 to 183 (10)
File: redis.go
    noCacheLookupURL Lines: 19 to 20 (1)
    noCacheDeactivateURL Lines: 20 to 21 (1)
    noCacheSavePopularURL Lines: 21 to 29 (8)
    CreateRedisCache Lines: 29 to 58 (29)
    CreateRedisCachefunc1 Lines: 36 to 37 (1)
    RedisCacheLookupURL Lines: 58 to 110 (52)
    RedisCacheDeactivateURL Lines: 82 to 93 (11)
    RedisCacheSavePopularURL Lines: 93 to 118 (25)
    hasURL Lines: 104 to 110 (6)
File: router.go
    init1 Lines: 15 to 22 (7)
    AssetsRewrite Lines: 22 to 44 (22)
    AssetsRewritefunc1 Lines: 23 to 67 (44)
    glob.func1 Lines: 34 to 36 (2)
    locateHandler Lines: 44 to 66 (22)
    RegisterHandlers Lines: 66 to 81 (15)
    RegisterHandlersfunc1 Lines: 67 to 83 (16)
    Recover Lines: 81 to 94 (13)
    Recoverfunc1 Lines: 82 to 95 (13)
    Recover.func11 Lines: 83 to 85 (2)
    AdminAuth Lines: 94 to 107 (13)
    AdminAuthfunc1 Lines: 95 to 102 (7)
    validateAdminToken Lines: 107 to 122 (15)
File: util.go
    RandomString Lines: 17 to 33 (16)
    BodyMerge Lines: 33 to 43 (10)
    JSON Lines: 43 to 49 (6)

[bingoohuang]

延长变量的生命周期 runtime.KeepAlive(v)

2019年的一篇文章：https://medium.com/a-journey-with-go/go-keeping-a-variable-alive-c28e3633673a

package main

import (
    "io/ioutil"
    "log"
    "os"
    "runtime"
    "syscall"
)

type File struct{ d int }

func main() {
    file, err := ioutil.TempFile("", "keepalive")
    if err != nil {
        log.Fatal(err)
    }
    file.Write([]byte("keepalive"))
    file.Close()
    defer os.Remove(file.Name())

    p := openFile(file.Name())
    content := readFile(p.d)

    // Ensure p is not finalized until Read returns
    // runtime.KeepAlive(p)

    println("Here is the content: " + content)
}

func openFile(path string) *File {
    d, err := syscall.Open(path, syscall.O_RDONLY, 0)
    if err != nil {
        panic(err)
    }

    p := &File{d}
    runtime.SetFinalizer(p, func(p *File) {
        syscall.Close(p.d)
    })

    return p
}

func readFile(descriptor int) string {
    doSomeAllocation()

    var buf [1000]byte
    _, err := syscall.Read(descriptor, buf[:])
    if err != nil {
        panic(err)
    }

    return string(buf[:])
}

func doSomeAllocation() {
    var a *int

    // memory increase to force the GC
    for i := 0; i < 10000000; i++ {
        i := 1
        a = &i
    }

    _ = a
}

输出:

panic: device not configured

goroutine 1 [running]:
main.readFile(0x3, 0x43, 0xc000120010)
        /Users/bingoo/GitHub/gogotcha/cmd/keepalive/main.go:51 +0x138
main.main()
        /Users/bingoo/GitHub/gogotcha/cmd/keepalive/main.go:23 +0x176

加上runtime.KeepAlive后（放开对应的注释行），

Here is the content: keepalive

[bingoohuang]

Go的 50 度灰：Golang 新开发者要注意的陷阱和常见错误

50 度灰，先看书后观影

1. Go的 50 度灰 原版 part 1
2. Go的 50 度灰 原本 part 2
3. Go的50度灰：开发者要注意的陷阱和常见错误

《五十度灰》（英语：Fifty Shades of Grey）是一部2015年上映的美国情色爱情电影，根据EL·詹姆丝的同名小说改编而成。由萨姆·泰勒-约翰逊执导，凯利·马塞尔编剧，并由达科塔·约翰逊、詹米·多南、珍妮佛·艾莉以及马西雅·盖·哈登主演[6]。电影于2015年2月11日在柏林国际电影节上首映[7]，2月13日正式公映。票房获得即时的佳绩，打破了多个票房纪录，全球票房5.71亿美元。

[bingoohuang]

用组合实现继承

继承，直达问题的本质，清晰易懂

type Foo struct {
     Base // 继承
     ...
}

type Foo struct {
     *Base // 虚拟继承
     ...
}

来源，许式伟， Go vs. GoPlus(Go+) 2021-6-27 北京 GopherChina2021

[bingoohuang]

介绍了 go 中dig和wire两个DI工具。其中dig是通过运行时反射实现的依赖注入。 而wire是根据自定义的代码，通过命令，生成相应的依赖注入代码，在编译期就完成依赖注入，无需反射机制。 这样的好处是：

1. 方便排查，如果存在依赖错误，编译时就能发现。而 dig 只能在运行时才能发现依赖错误。
2. 避免依赖膨胀，wire生成的代码只包含被依赖的，而dig可能会存在好多无用依赖。
3. 依赖关系静态存在源码，便于工具分析。

为什么把 dig 迁移到 wire

[bingoohuang]

为什么腾讯越来越倾向于使用Go语言？

原因可能会有很多，关于Go语言的特性、优势等。但是最主要的原因，应该是基于两方面的考虑：执行性能&开发效率。

Go 语言以其接近 C 的执行性能和近解释型语言的开发效率，以及近乎于完美的编译速度，已经广泛应用于人工智能、云计算开发、容器虚拟化、大数据开发、 数据分析及科学计算、运维开发、爬虫、游戏开发等领域。

Go语言适用于各个领域行业-前景广阔

[bingoohuang]

Sealed Interfaces 密封接口

看到一种 go 代码，接口里定义一个私有无参无返回值的方法，感觉是一种惯用法，原来是密封接口，保证这个接口不给外部去实现。

// Statement represents a statement.
type Statement interface {
    iStatement()
    SQLNode
}

func (*Union) iStatement()         {}
func (*Select) iStatement()        {}
func (*Insert) iStatement()        {}
func (*Update) iStatement()        {}
func (*Delete) iStatement()        {}
func (*Set) iStatement()           {}
func (*DDL) iStatement()           {}
func (*Show) iStatement()          {}
func (*Use) iStatement()           {}
func (*OtherRead) iStatement()     {}
func (*OtherAdmin) iStatement()    {}
func (*TruncateTable) iStatement() {}

同样的

// SelectStatement any SELECT statement.
type SelectStatement interface {
    iSelectStatement()
    iStatement()
    iInsertRows()
    AddOrder(*Order)
    SetLimit(*Limit)
    SQLNode
}

func (*Select) iSelectStatement()      {}
func (*Union) iSelectStatement()       {}
func (*ParenSelect) iSelectStatement() {}

[bingoohuang]

所有模型都是错误的，但有些模型是有用的

在 go-profiler-notes 上看到一句话： All models are wrong

本质上，所有模型都是错误的，但有些模型是有用的。

--- Box，乔治·EP；Norman R.Draper（1987）。经验模型的建立和响应面，p。424，威利。ISBN 0471810339。

我认为最好通过两部分来分析它的含义：

“所有模型都是错误的”，也就是说，每个模型都是错误的，因为它是对现实的简化。一些模型，特别是在“硬”科学中，只是有点错误。他们无视摩擦或微小物体的引力作用。其他模型有很多错误-他们忽略了更大的事情。在社会科学中，我们忽略了很多。

“但是有些有用”-简化现实可能非常有用。它们可以帮助我们解释，预测和理解宇宙及其所有组成部分。

这不仅在统计上是正确的！地图是一种模型。他们错了。但是好的地图非常有用。其他有用但错误的模型的例子比比皆是。

比例为1：1的完美地图的幻想已被许多作者使用，包括Lewis Carroll，Jorge Luis Borges和Umberto Eco。实际上，这没有用，因为它所映射的区域必然很复杂，而且不容易理解（更不用说将其展开并布置为阅读的尴尬了）。

也许您还可以补充一点，就是模型必须有点错误，因为否则它将无法泛化，因此无法在其他地方应用。有一些答案说明了这一点。但是现在有太多的答案无法全部阅读。

对我来说，真正的见解在于以下方面：

模型不一定非要正确才有用。

不幸的是，在许多科学中，人们常常忘记了模型不一定必须是现实的精确表示即可允许新的发现和预测！

因此，不要浪费您的时间来构建需要对无数变量进行准确测量的复杂模型。真正的天才发明了可以完成这项工作的简单模型。

[bingoohuang]

unsafe.Pointer and uintptr

unsafe.Pointer and uintptr

package main

import "unsafe"

func f() {
    var p uintptr
    for i := range [10]int{} {
        var x = i
        if i == 0 {
            // uintptr类型的临时变量只是一个普通的数字，所以其值不应该被改变。
            // 因此，x 变量在栈上，10次循环，所使用的栈地址不变，因此 p 最后的值是9
            p = uintptr(unsafe.Pointer(&x))
        }
    }
    println(*(*int)(unsafe.Pointer(p))) // 9
}

func g() {
    var p unsafe.Pointer
    for i := range [10]int{} {
        var x = i
        if i == 0 {
            // 从gc视角来看，unsafe.Pointer是一个指向变量的指针，因此当变量被移动是对应的指针也必须被更新
            // 为了跟踪指针更新， x 逃逸到了堆上，10次循环，生成了10个不同的x堆变量， p指向了第1个，其值是0
            p = unsafe.Pointer(&x)
        }
    }
    println(*(*int)(p)) // 0
}

func main() {
    f()
    g()
}

$ go run -gcflags="-m" p.go
# command-line-arguments
./p.go:19:7: moved to heap: x
9
0

从垃圾收集器的视角来看，一个unsafe.Pointer是一个指向变量的指针，因此当变量被移动是对应的指针也必须被更新；但是uintptr类型的临时变量只是一个普通的数字，所以其值不应该被改变。

Go语言圣经（中文版）

此例，即考察了unsafe.Pointer,uintptr，又考察了变量逃逸。

[bingoohuang]

程序应该是为人们阅读而编写的，只是偶然地供机器执行—— Abelson 和 Sussman

A先生和S先生，是谁，谷歌一下，原来都是大神啊，大神说的话，应该选择相信。

• 问：“如何组织 Go 代码？”。
• 答：“我不知道。” ，设计代码时，总是从扁平结构开始，并在必要时创建包。 -- Robert Griesemer（Go 的作者之一）

1. How to structure Go code?，如何组织 Go 代码？Go 作者的回答惊呆了

我摘抄在这里，是想说，我深有同感。我总是从单一的 main.go 开始，随着功能的增加，再分拆结构，也可能一个 main.go 文件就可以了，特别是在制作一些小工具的时候。

[bingoohuang]

go1.18 泛型变快了嘛？

Credit: https://unsplash.com/photos/CpkOjOcXdUY

看博客Go is about to get a whole lot faster

源代码：

1. deque 无泛型版本
2. deque 有泛型版本
3. 二者差异

操作步骤：

1. cd deque-generic && go1.18beta1 test -run=NONE -bench=. > ../generic.txt
2. cd deque-non-generic && go1.18beta1 test -run=NONE -bench=. > ../none.txt
3. cd deque-non-generic && go test -run=NONE -bench=. > ../none-go1.17.6.txt

结果输出：

$ go install golang.org/x/tools/cmd/benchcmp@latest   
➜  deque more generic.txt 
goos: darwin
goarch: amd64
pkg: github.com/sekoyo/deque
cpu: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
BenchmarkPushFront-12           97286768                12.28 ns/op
BenchmarkPushBack-12            142144150               14.43 ns/op
BenchmarkSerial-12              100000000               11.82 ns/op
BenchmarkSerialReverse-12       100000000               11.21 ns/op
BenchmarkRotate-12                 56238            120075 ns/op
BenchmarkInsert-12                 36441            120364 ns/op
BenchmarkRemove-12                107545            119688 ns/op
BenchmarkYoyo-12                    1777            675390 ns/op
BenchmarkYoyoFixed-12               2662            455608 ns/op
PASS
ok      github.com/sekoyo/deque 33.914s
➜  deque more non.txt    
goos: darwin
goarch: amd64
pkg: github.com/gammazero/deque
cpu: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
BenchmarkPushFront-12           25286799                50.48 ns/op
BenchmarkPushBack-12            28099020                41.14 ns/op
BenchmarkSerial-12              25631952                50.65 ns/op
BenchmarkSerialReverse-12       26357466                50.56 ns/op
BenchmarkRotate-12                 40188            119641 ns/op
BenchmarkInsert-12                 28888            120136 ns/op
BenchmarkRemove-12                 87488            123670 ns/op
BenchmarkYoyo-12                     574           2039762 ns/op
BenchmarkYoyoFixed-12                877           1382135 ns/op
PASS
ok      github.com/gammazero/deque      28.086s
➜  deque benchcmp none.txt generic.txt 
benchcmp is deprecated in favor of benchstat: https://pkg.go.dev/golang.org/x/perf/cmd/benchstat
benchmark                     old ns/op     new ns/op     delta
BenchmarkPushFront-12         50.5          12.3          -75.67%
BenchmarkPushBack-12          41.1          14.4          -64.92%
BenchmarkSerial-12            50.6          11.8          -76.66%
BenchmarkSerialReverse-12     50.6          11.2          -77.83%
BenchmarkRotate-12            119641        120075        +0.36%
BenchmarkInsert-12            120136        120364        +0.19%
BenchmarkRemove-12            123670        119688        -3.22%
BenchmarkYoyo-12              2039762       675390        -66.89%
BenchmarkYoyoFixed-12         1382135       455608        -67.04%
➜  deque more none-go1.17.6.txt 
goos: darwin
goarch: amd64
pkg: github.com/gammazero/deque
cpu: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
BenchmarkPushFront-12           24497521                50.09 ns/op
BenchmarkPushBack-12            32226673                39.35 ns/op
BenchmarkSerial-12              28485412                46.10 ns/op
BenchmarkSerialReverse-12       27771453                46.76 ns/op
BenchmarkRotate-12                 40466            121028 ns/op
BenchmarkInsert-12                 28731            121774 ns/op
BenchmarkRemove-12                 86724            125651 ns/op
BenchmarkYoyo-12                     620           2049784 ns/op
BenchmarkYoyoFixed-12                820           1389294 ns/op
PASS
ok      github.com/gammazero/deque      28.335s

还真是，在 PushFront 上竟然快了 75%，看来等 1.18 正式发布了，还是要香一下的。

[bingoohuang]

Go 可执行文件大小 树图分析

go-binsize-treemap

1. go install github.com/nikolaydubina/go-binsize-treemap@latest
2. go tool nm -size <binary finename> | go-binsize-treemap > binsize.svg

[bingoohuang]

在线火焰图

https://flamegraph.com/

Pyroscope Go Playground

[bingoohuang]

有名返回参数的坑

1. https://mp.weixin.qq.com/s/RpeiByFggXal07awqfT8vA
2. https://twitter.com/bwplotka/status/1494362886738780165

Bartłomiej Płotka 出的题

package main

func aaa() (done func(), err error) {
    return func() {
        print("aaa: done")
    }, nil
}

func bbb() (done func(), _ error) {
    done, err := aaa()
    return func() {
        print("bbb: surprise!")
        done()
    }, err
}

func main() {
    done, _ := bbb()
    done()
}

这个程序输出结果是什么呢，（单选）？

• [ ] 1. 输出 aaa: done
• [ ] 2. 输出 ​bbb: surprise!aaa: done
• [ ] 3. 一直输出，永远不结束
• [ ] 4. 程序最终运行出错

正确答案是：D，程序最终运行出错。一直调用一直爽，直至栈溢出程序崩溃。

他会不断地递归，疯狂输出 “bbb: surprise!”，直至栈溢出，导致程序运行出错，最终中止。

同学就疑惑了，怎么又多出了个递归？

本质上在函数 bbb 执行完毕后， 变量 done 已经变成了一个递归函数。

递归的过程是：函数 bbb 调用变量 done 后，会输出 bbb: surprise! 字符串，然后又调用变量 done。而变量 done 又是这个闭包（匿名函数），从而实现不断递归调用和输出。

总结

这位大佬出的题目，本质上是比较烦人的，其结合了函数返回参数的命名用法。

如果我们把这个函数的返回参数命名去掉，就可以避开这个问题。

[bingoohuang]

GO 语言纪录片

Go: A Documentary

[bingoohuang]

详解 Go 中的 rune 类型

https://mp.weixin.qq.com/s/hcrq5fYaQ7FN_2oSMRNjcA

Go 语言把字符分 byte 和 rune 两种类型处理。byte 是类型 unit8 的别名，用于存放占 1 字节的 ASCII 字符，如英文字符，返回的是字符原始字节。rune 是类型 int32 的别名，用于存放多字节字符，如占 3 字节的中文字符，返回的是字符 Unicode 码点值。如下图所示：

s := "Go语言编程"
// byte
fmt.Println([]byte(s)) // 输出：[71 111 232 175 173 232 168 128 231 188 150 231 168 139]
// rune
fmt.Println([]rune(s)) // 输出：[71 111 35821 35328 32534 31243]

[bingoohuang]

docker 编译

Dockerfile:

FROM golang:alpine AS builder
WORKDIR /build
RUN apk add upx
COPY . .
RUN go build -ldflags "-s -w" -o hello hello.go && upx hello

FROM alpine
WORKDIR /build
COPY --from=builder /build/hello /build/hello
CMD ["./hello"]

1. docker build -t hello:v1 .
2. dive hello:v1 .

$ docker run -it --rm hello:v1
hello world!
$ docker run -it --rm hello:v1 ls -lh /build
total 332K
-rwxr-xr-x    1 root     root      331.2K Mar 15 02:12 hello
$ docker images | grep hello
hello        v1        b3762d8a6c76   12 minutes ago   5.92MB

[bingoohuang]

源代码特洛伊木马攻击

1. https://go.dev/play/p/e2BDZvFlet0
2. https://coolshell.cn/articles/21649.html

~/aaa ❯ go install github.com/breml/bidichk/cmd/bidichk@latest
~/aaa 38s ❯ more a.go                                                                                        14:49:52
package main

import "fmt"

func main() {
        str, mask := "Hello, World!<U+202E>10x<U+202D>", 0

        bits := 0
        for _, ch := range str {
                for ch > 0 {
                        bits += int(ch) & mask
                        ch = ch >> 1
                }
        }
        fmt.Println("Total bits set:", bits)
}

~/aaa ❯ bidichk a.go                                                                                         14:49:56
/Users/bingoobjca/aaa/a.go:6:29: found dangerous unicode character sequence RIGHT-TO-LEFT-OVERRIDE
/Users/bingoobjca/aaa/a.go:6:35: found dangerous unicode character sequence LEFT-TO-RIGHT-OVERRIDE

https://goplay.tools/snippet/2Ty5h65xSjv

package main

import "fmt"

func main() {
    str := "Hello, World!‮1234567‭890"
    fmt.Println(str)
}

Output:

Hello, World!‮1234567‭890

[garfield-su]

package main

import "fmt"

func main() {
    //str, mask := "Hello, World!‮10x‭", 0
    str, mask := "Hello, World!<U+202E>10x<U+202D>", 0

    bits := 0
    for _, ch := range str {
        for ch > 0 {
            bits += int(ch) & mask
            ch = ch >> 1
        }
    }
    fmt.Println("Total bits set:", bits)
    str1 := "Hello, World!‮1234567‭8901               ;"
    fmt.Println(str1)
    str2 := "Hello, World!‮1234567‭8901                "
    fmt.Println(str2)
}

result

Total bits set: 0 Hello, World!‮1234567‭8901 ; Hello, World!‮1234567‭8901

Program exited.

[bingoohuang]

为什么是 int 类型

• code

package main

import (
    "fmt"
)

func main() {
    v := 4
    fmt.Printf("%T\n", v)

    f := ((v / 5.0) * 100.0) * 11.0 / 100.0
    fmt.Printf("Type of result %T for value %0.2f\n", f, f)

    fmt.Printf("\n%T\n", 4)
    f2 := ((4 / 5.0) * 100.0) * 11.0 / 100.0
    fmt.Printf("Type of result %T for value %0.2f\n", f2, f2)
}

输出结果：

int
Type of result int for value %!f(int=00)

int
Type of result float64 for value 8.80

GO SSA

[bingoohuang]

GO 从源代码到可执行文件的编译过程：

连接时，会打上运行时

Hello World, from the code to the screen

[bingoohuang]

https://mp.weixin.qq.com/s/bg3LarfaiSKvs5U4mgnEEg

Go Team为Gopher负重前行！

这是在X.com上看到一幅图片，从中可以看出Go team为Gopher的“just write business logic”而负重前行。

就像我在《Go语言精进之路》第一卷中所说的那样：“Go的设计者们在语言设计之初就拒绝走语言特性融合的道路，而选择了“做减法”。并且他们把复杂性留给了语言自身的设计和实现，留给了Go Team自己；而将简单、易用和清晰留给了广大Gopher”。让广大Gopher可以用简单的语法聚焦于编写business logic的代码上。