cmd/go: add *_unix.go recognition (and presumably unix build tag)

[robpike]

It does understand _windows.go, and I was surprised to (re)learn that it does not support _unix.go. Would it do so, things would be clearer and all the *_unix.go files that already exist wouldn't need to maintain their

// +build darwin dragonfly freebsd linux netbsd openbsd solaris

tags as the list grows.

[bradfitz]

More concretely, does this mean "unix" would mean anything in a pre-defined list of GOOS values? Would we proactively add unsupported GOOS values that might be supported later, like we did with GOARCH values, or would "unix" mean anything that's not windows or plan9 or nacl? Wait, is nacl unix?

Also, should we define "posix"? Currently we have some _posix.go files which include windows.

I don't object to this, but I'm just curious which definition we'd use.

[robpike]

All good questions for which I don't know the answer.

My main concern is maintaining the long list of os names in the build tags. Making things easier would be an advantage too.

Discussion needed.

[j7b]

I'd suggest the focus should be on user code and the standard library, and a new build tag is required (perhaps "unix"). The standard library has clear manifestations of a notion of "unix" in packages intended for general use (most obviously in net), and I'd suggest any GOOS that implements these should satisfy the build constraint word "unix." For example, if nacl's net package supports network types "unix", "unixgram" and "unixpacket" it would satisfy some of the predicates the "unix" build constraint should require.

The *_unix.go convention could implicitly define +build unix.

[robpike]

That's pretty much what I mean.

[dominikh]

Related: #6325

[jimmyfrasche]

Looking at the list of OSes, the BSD's may be the most homogeneous and largest sublist. But do you count Solaris, Darwin? Even if not, just defining bsd as dragonfly freebsd netbsd openbsd seems handy regardless of any other tags that might be included. If Darwin and Solaris are included, the build tag in the OP shrinks down to // +build linux bsd

Looking at the feature sets, posix may be the simplest target, but which subset of posix? Is it just the shared subset of all the unix-like OSes currently supported? When a new posix-y OS is added does that mean it can't be added to posix if it's missing one thing the other posix OSes support or do the set of guaranteed features lose a member? Since posix is so vast and as far as I know in practice little agreed upon, that seems the most problematic.

Looking at the stdlib, there seems to be very little agreement in practice among which OSes to include, even in *_unix.go and *_posix.go files. Of course, that doesn't preclude having build tags like // +build unix,!freebsd when needed.

Looking to the future, if the list is not proactive, as Brad put it, that means a new OS could suddenly match with a new version of Go when it should not. That may be worrisome but then again currently we have the situation where a new GOOS doesn't match anything without manual intervention. Of course, even a future proofed list is only as good as targets currently known and can't contain as yet uninvented OSes, and then the same issues arise as with posix.

[j7b]

From a user code perspective it shouldn't matter if the "unix" is bsd, lunix, solaris or other. I'd consider mandating cgo for test purposes for "unix" candidates and having a cgo-centric test suite to make sure all the unix-type-things for a given Go toolchain aren't at odds with the C toolchain as known to the Go toolchain for a given GOOS.

[rsc]

This is basically a straight duplicate of #6325, as @dominikh points out. The problem here, as there, is that "unix" changes based on various details - sometimes it includes Linux, sometimes not, for example. It's not clear we want to be in the business of defining it exactly one way. Also, this should not affect users terribly much, given that we try to provide OS-independent APIs to the largest extent possible.

[robpike]

You're skirting the point here, which is just to avoid maintaining the huge list of Unix variants Go supports. It's to discriminate the basic difference between Windows and Unix. Windows has many versions, but there is only one predefined "windows" tag, in the sense the foo_windows.go works. I'm asking for the same courtesy for Unix.

[jimmyfrasche]

Ignoring ios and android for the moment, as they're essentially defined to be "variants" of darwin and linux, and only looking at the GOOS build tags, the complete list, as of 1.8, is:

windows plan9 nacl linux solaris darwin dragonfly openbsd netbsd freebsd

The goal, as I understand it, is to be able to use *_GOOS.go as much as possible, by defining one or more "super-GOOS" tags that match a large subset of the above tags.

I'm going to ignore the implications of defining unix or posix and just look at what's in the stdlib so there's some data. (Apologies if I made any mistakes).

Looking at the *_posix.go files in the $GOROOT/src† the definition of posix uses seems to be almost equivalent to !plan9, with occasion to additionally specify !nacl (3/15 of the files) and once adding !solaris.

(Windows is not what most people think of when they thing posix. An alternative would be to allow something like x_not-T.go to be equivalent to a // +build !T directive and rename most of those as _not-plan9.go.)

† using

find -type f -name "*_posix.go" -exec grep "^// [+]build " {} \; | cut -d" " -f3- | sort

For the *_unix.go files,I took the list of tags, removed two that, despite the name, only had to do with cgo, took out all arches, android build tags, etc., manually expanded the negations, reordered them so all the tags were in the same order and then counted the occurrences of each unique subset of GOOSes:

18: darwin dragonfly freebsd linux nacl netbsd openbsd solaris
12: darwin dragonfly freebsd linux netbsd openbsd solaris
3: darwin dragonfly freebsd linux nacl netbsd openbsd plan9 solaris
2: darwin dragonfly freebsd linux netbsd openbsd
2: darwin dragonfly freebsd linux nacl netbsd openbsd solaris windows
1: dragonfly freebsd linux nacl netbsd openbsd solaris
1: dragonfly freebsd linux netbsd openbsd
1: dragonfly linux netbsd openbsd solaris
1: freebsd linux netbsd

Looking at that it seems !windows,!plan9 would get the most existing *_unix.go files, fit most people's expected definition, but often require an additional // +build !nacl directive.

In practice how build tags are written seems to be somewhat ad hoc. Maybe gofmt could sort them in a standard way or golint warn if they're not alphabetical? Perhaps a new tool that deals solely with build constraints?

[jimmyfrasche]

Here's similar for every GOOS build tag : https://gist.github.com/jimmyfrasche/24f45e37a35f02eab8dcd1fb2c1c6af2

[rsc]

It sounds like the real interesting use would be outside the standard library. Can someone survey what usages look like outside, like in the go corpus files?

[j7b]

Maybe the best expression of "unix" is !windows,!android,!plan9,!nacl as it's much easier to express why a GOOS is !unix and the build tag waters aren't muddied when user code has to pander to particular variants of things that aren't !unix.

[rsc]

On hold for someone to present evidence about the need for this outside the standard library.

[robpike]

Any code that needs to support variant operating systems would need this. For example, Upspin has

errors_unix.go:5:// +build darwin linux freebsd netbsd

Is that the right list? No. What is? I don't know, and the code will go out of date as new variants arise. The code in that file is Unix-generic, but what is Unix from Go's point of view?

It seems to me that if we can define golang.org/x/sys/unix then we can define build tag unix and _unix.go.

[vmarkovtsev]

I have gathered some stats on the GitHub dataset in BigQuery.

First I extracted all the "// +build" lines:

SELECT C.id as id, MIN(C.content) as content, MIN(F.path) as path, MIN(C.size) as size, MIN(F.repo_name) as repo
FROM `bigquery-public-data.github_repos.contents` as C
JOIN `bigquery-public-data.github_repos.files` as F
ON C.id = F.id AND F.path LIKE '%.go'
GROUP BY id

Query complete (251.7s elapsed, 2.26 TB processed)

SELECT line, COUNT(*) as n
FROM (
  SELECT SPLIT(content, '\n') as line
  FROM [eminent-airport-191509:github.go_content]
  HAVING LEFT(line, 9) = '// +build'
)
GROUP BY line
ORDER BY n DESC

Query complete (10.0s elapsed, 11.2 GB processed)

The raw result is attached as lines.csv.gz

Then I calculated the cooccurrences using a Neugram script bake.txt and the result is attached as result.csv.

Finally, I normalized the co-occurrence matrix by column (sum=1) and plotted it using a Python script:

How to interpret: OS Y -> OS X cell is whiter if X occurs with Y more often than with the rest of the column. E.g. Plan9 is mentioned together with Windows more often than with anything else. We can see that BSDs, Linux and Solaris tend to go together.

[trex58]

There is a new GOOS value to take into account now: aix . We have ported GCC Go on AIX 7 and we are working now on porting golang. Nearly all changes dealing with AIX and GCC Go have been submitted and merged. We are now working on golang.org/x and it appears that it contains a lot of OS cases. Moreover, colleagues who are porting Go applications on AIX are facing many cases where they have to manage the OS (adding aix) though the code is far from the Operating System.

About golang.org/x, here are some recent (2018/05/30) data about how it depends on "linux", goos, and goarch . That provides an idea about the amount of work required for adding a new operating system or a new architecture.

Number of times the expression: '// +build ...linux...' appears in the sources of each package: arch: 0 benchmarks: 3 blog: 0 build: 3 crypto: 8 exp: 14 image: 0 mobile: 34 net: 54 oauth2: 0 perf: 0 sync: 0 sys: 94 talks: 0 text: 0 tools: 8 vgo: 5

Number of times the key-word: 'linux' appears in the sources of each package: arch: 1 benchmarks: 3 blog: 5 build: 488 crypto: 9 exp: 18 image: 0 mobile: 57 net: 153 oauth2: 0 perf: 12 sync: 0 sys: 283 talks: 15 text: 0 tools: 46 vgo: 70

Number of times the key-word: 'goos' appears in the sources of each package: arch: 0 benchmarks: 1 blog: 1 build: 35 crypto: 0 exp: 0 image: 0 mobile: 19 net: 0 oauth2: 0 perf: 10 sync: 0 sys: 8 talks: 0 text: 0 tools: 6 vgo: 27

Number of times the key-word: 'goarch' appears in the sources of each package: arch: 0 benchmarks: 1 blog: 0 build: 37 crypto: 0 exp: 0 image: 0 mobile: 8 net: 0 oauth2: 0 perf: 10 sync: 0 sys: 5 talks: 0 text: 0 tools: 6 vgo: 28

[mortdeus]

dang I thought this was going to be a proposal to port Go to run on the old PDP unix made at bell labs. :(

Also, I explored thinking about this awhile back but how hard would it be to port Go to run on top of a virtual OS layer like inferno?

[rsc]

On Jun 5 2017 I wrote:

On hold for someone to present evidence about the need for this outside the standard library.

Issue #38364 and #37503 may be evidence that the standard library is need enough, because it would make it easier to maintain Unix-like ports outside the main Go tree.

Perhaps it would be enough to define unix = all the Unixes and then systems that want to use a unix tag can filter with things like

// +build unix
// +build !linux

(There's a proposal to make these better. Still not ready to post that yet.)

[rsc]

There are some stats above, but they might be old. Does anyone want to investigate how much things might get cleaned up for code outside the main tree if we added "unix"?

For sake of discussion, unix would be all the operating systems traditionally recognized as Unix (aix, android, darwin, dragonfly, freebsd, hurd, illumos, linux, netbsd, openbsd, solaris; NOT js, nacl, plan9, windows, zos). It is of course ironic that GNU Hurd is unix by this definition.

[Xuanwo]

go-locale is a lib for cross platform locale detection, we have to add build tags like following:

https://github.com/Xuanwo/go-locale/blob/master/locale_unix.go#L1

[smasher164]

I did a search with the sourcegraph cli across repos outside the main tree for files that have the suffix _unix.go.

The command I ran to query and fetch the data:

$ src search -json -- '-file:vendor/ -file:node_modules/ lang:go -repo:github.com/golang/go file:.*_unix.go count:1000' > out.json

The command I ran to list the unique repositories:

$ gron out.json | awk -F'"' '/.repository.name/{print $2}' | sort | uniq

Output

github.com/99designs/aws-vault
github.com/AdguardTeam/AdGuardHome
github.com/Azure/draft
github.com/GoogleContainerTools/skaffold
github.com/TarsCloud/TarsGo
github.com/adnanh/webhook
github.com/aerokube/selenoid
github.com/alecthomas/gometalinter
github.com/alecthomas/kingpin
github.com/bettercap/bettercap
github.com/boltdb/bolt
github.com/boyter/scc
github.com/browsh-org/browsh
github.com/btcsuite/btcd
github.com/cloud-custodian/cloud-custodian
github.com/cloudflare/cfssl
github.com/cockroachdb/cockroach
github.com/concourse/concourse
github.com/containerd/containerd
github.com/containernetworking/cni
github.com/containers/buildah
github.com/containous/traefik
github.com/cosmos72/gomacro
github.com/coyove/goflyway
github.com/cyfdecyf/cow
github.com/datawire/ambassador
github.com/dgraph-io/badger
github.com/dgraph-io/dgraph
github.com/docker/classicswarm
github.com/docker/cli
github.com/docker/docker-ce
github.com/docker/machine
github.com/dominikh/go-tools
github.com/elastic/beats
github.com/eolinker/goku-api-gateway
github.com/etcd-io/bbolt
github.com/etcd-io/etcd
github.com/ethereum/go-ethereum
github.com/fabiolb/fabio
github.com/fluxcd/flux
github.com/flynn/flynn
github.com/gcc-mirror/gcc
github.com/getlantern/lantern
github.com/ginuerzh/gost
github.com/go-delve/delve
github.com/go-gitea/gitea
github.com/go-vgo/robotgo
github.com/golang/dep
github.com/golang/tools
github.com/google/cadvisor
github.com/google/gopacket
github.com/google/syzkaller
github.com/gopherjs/gopherjs
github.com/gravitational/teleport
github.com/gruntwork-io/terragrunt
github.com/hajimehoshi/ebiten
github.com/hashicorp/consul
github.com/hashicorp/consul-template
github.com/hashicorp/nomad
github.com/hashicorp/packer
github.com/hashicorp/terraform
github.com/hashicorp/vault
github.com/helm/helm
github.com/iikira/BaiduPCS-Go
github.com/influxdata/influxdb
github.com/influxdata/telegraf
github.com/ipfs/go-ipfs
github.com/itchyny/bed
github.com/jpmorganchase/quorum
github.com/junegunn/fzf
github.com/keybase/client
github.com/koding/kite
github.com/kubernetes-retired/contrib
github.com/kubernetes/kubernetes
github.com/lileio/lile
github.com/linkedin/Burrow
github.com/linuxkit/linuxkit
github.com/minio/minio
github.com/minishift/minishift
github.com/mit-pdos/biscuit
github.com/moby/buildkit
github.com/moby/moby
github.com/motemen/gore
github.com/mozilla/sops
github.com/muesli/beehive
github.com/mvdan/sh
github.com/ochinchina/supervisord
github.com/odeke-em/drive
github.com/oklog/oklog
github.com/onsi/ginkgo
github.com/opencontainers/runc
github.com/ouqiang/gocron
github.com/oxequa/realize
github.com/pachyderm/pachyderm
github.com/peco/peco
github.com/perkeep/perkeep
github.com/perlin-network/life
github.com/photoprism/photoprism
github.com/portainer/portainer
github.com/prometheus/prometheus
github.com/pulumi/pulumi
github.com/rancher/rancher
github.com/rclone/rclone
github.com/restic/restic
github.com/segmentio/chamber
github.com/shirou/gopsutil
github.com/sirupsen/logrus
github.com/smallnest/rpcx
github.com/spf13/afero
github.com/syncthing/syncthing
github.com/syndtr/goleveldb
github.com/theupdateframework/notary
github.com/tidwall/evio
github.com/tinygo-org/tinygo
github.com/upspin/upspin
github.com/v2ray/v2ray-core
github.com/valyala/fasthttp
github.com/variadico/noti

The command I ran to list the unique build tags (ordered by frequency):

$ gron out.json | awk -F'"' '/file.content/{$1="";printf "%s%c", $0, 0}' | xargs -0 printf "%b" | awk -F"build " '/\+build/{print $2}' | sort | uniq -c | sort -nr

Output

 303 !windows
  44 linux freebsd
  34 darwin dragonfly freebsd linux netbsd openbsd solaris
  24 darwin dragonfly freebsd linux nacl netbsd openbsd solaris
  14 darwin dragonfly freebsd linux netbsd openbsd
  14 aix darwin dragonfly freebsd hurd linux netbsd openbsd solaris
  14 aix darwin dragonfly freebsd hurd js,wasm linux netbsd openbsd solaris
   8 linux freebsd darwin
   7 linux darwin
   7 aix darwin dragonfly freebsd linux netbsd openbsd solaris
   7 !windows,!plan9
   7 !windows,!darwin
   6 linux darwin freebsd
   5 freebsd darwin
   5 !windows,!nacl,!plan9
   5 !windows,!linux
   5 !appengine
   4 linux darwin freebsd openbsd netbsd
   4 dragonfly freebsd linux netbsd openbsd solaris
   4 darwin freebsd linux netbsd openbsd
   4 darwin freebsd linux
   4 cgo
   4 !linux,!windows
   3 linux freebsd darwin openbsd
   3 linux darwin dragonfly freebsd openbsd netbsd solaris
   3 linux
   3 freebsd linux darwin
   3 freebsd linux
   3 !windows,!plan9,!linux,!openbsd
   3 !plan9,!windows
   3 !nofuse
   3 !linux,!darwin,!freebsd
   3 !js
   3 !android
   2 linux netbsd openbsd solaris
   2 linux freebsd openbsd darwin solaris illumos
   2 linux freebsd openbsd
   2 linux aix
   2 freebsd openbsd netbsd dragonfly darwin linux solaris
   2 dragonfly freebsd linux nacl netbsd openbsd solaris
   2 darwin linux,!baremetal freebsd,!baremetal
   2 darwin freebsd solaris
   2 darwin dragonfly freebsd linux,!appengine netbsd openbsd
   2 darwin dragonfly freebsd linux,!386 netbsd openbsd
   2 darwin dragonfly freebsd hurd linux netbsd openbsd
   2 darwin dragonfly freebsd !android,linux netbsd openbsd solaris aix
   2 cgo,!osusergo
   2 cgo,!netgo
   2 aix darwin dragonfly freebsd hurd js linux netbsd openbsd solaris
   2 !plan9
   2 !nofuse,!openbsd,!netbsd
   1 netbsd freebsd dragonfly linux
   1 linux,cgo darwin,cgo
   1 linux,!appengine darwin freebsd netbsd openbsd
   1 linux,!appengine darwin freebsd
   1 linux solaris darwin freebsd
   1 linux solaris
   1 linux netbsd freebsd
   1 linux freebsd solaris openbsd
   1 linux freebsd netbsd openbsd dragonfly darwin
   1 linux freebsd netbsd openbsd
   1 linux darwin freebsd solaris
   1 linux darwin freebsd openbsd solaris netbsd
   1 linux darwin freebsd openbsd netbsd dragonfly
   1 linux darwin freebsd netbsd openbsd solaris
   1 linux darwin freebsd netbsd openbsd dragonfly
   1 linux darwin freebsd netbsd openbsd
   1 js,!windows
   1 integration,benchmark,!windows
   1 illumos
   1 freebsd,!appengine netbsd,!appengine openbsd,!appengine linux,!appengine darwin,!appengine
   1 freebsd linux netbsd openbsd solaris dragonfly
   1 freebsd linux netbsd
   1 freebsd darwin linux
   1 dragonfly linux netbsd openbsd freebsd solaris darwin
   1 dragonfly freebsd linux openbsd solaris
   1 dragonfly freebsd linux netbsd solaris
   1 dragonfly freebsd linux netbsd openbsd
   1 dragonfly freebsd !android,linux netbsd openbsd
   1 dragonfly darwin freebsd hurd !android,linux netbsd openbsd
   1 dragonfly darwin freebsd !android,linux netbsd openbsd
   1 darwin,386 darwin,amd64 dragonfly freebsd linux,!android nacl netbsd openbsd solaris
   1 darwin netbsd freebsd openbsd dragonfly linux
   1 darwin linux netbsd openbsd
   1 darwin linux
   1 darwin freebsd openbsd
   1 darwin freebsd linux netbsd openbsd solaris
   1 darwin freebsd linux netbsd
   1 darwin dragonfly freebsd netbsd openbsd
   1 darwin dragonfly freebsd linux netbsd openbsd solaris nacl
   1 darwin dragonfly freebsd linux nacl netbsd openbsd solaris windows
   1 darwin dragonfly freebsd linux nacl netbsd openbsd plan9 solaris
   1 darwin dragonfly freebsd linux nacl netbsd openbsd
   1 darwin dragonfly freebsd linux
   1 darwin dragonfly freebsd hurd linux netbsd openbsd solaris
   1 darwin dragonfly freebsd !android,linux netbsd openbsd solaris
   1 darwin dragonfly freebsd !android,linux nacl netbsd openbsd solaris
   1 darwin
   1 android darwin dragonfly freebsd linux netbsd openbsd solaris
   1 aix hurd linux darwin dragonfly freebsd openbsd netbsd solaris
   1 aix freebsd hurd linux netbsd
   1 aix dragonfly freebsd linux netbsd solaris
   1 aix dragonfly freebsd hurd js,wasm linux netbsd openbsd solaris
   1 aix darwin,386 darwin,amd64 dragonfly freebsd hurd linux,!android netbsd openbsd solaris
   1 aix darwin dragonfly linux netbsd openbsd solaris
   1 aix darwin dragonfly freebsd netbsd openbsd solaris
   1 aix darwin dragonfly freebsd linux nacl netbsd openbsd solaris
   1 aix darwin dragonfly freebsd hurd linux netbsd openbsd plan9 solaris
   1 aix darwin dragonfly freebsd hurd js,wasm linux netbsd openbsd solaris windows
   1 aix darwin dragonfly freebsd hurd js,wasm !android,linux netbsd openbsd solaris
   1 aix darwin dragonfly freebsd hurd !android,linux netbsd openbsd solaris
   1 !windows,!solaris,!plan9
   1 !windows,!plan9,!solaris,!linux
   1 !windows,!plan9,!solaris,!irix
   1 !windows,!plan9,!solaris,!aix
   1 !windows,!plan9,!solaris
   1 !windows,!plan9,!aix
   1 !windows,!openbsd,!netbsd,!nofuse
   1 !windows,!nofuse
   1 !windows,!linux android
   1 !windows,!freebsd,!dragonfly,!darwin
   1 !windows,!freebsd
   1 !windows,!darwin,!freebsd,!netbsd,!openbsd,!dragonfly,!solaris
   1 !windows,!darwin,!freebsd,!netbsd
   1 !windows,!android
   1 !solaris,!windows
   1 !solaris
   1 !openbsd
   1 !noplugin
   1 !netbsd
   1 !linux,!windows,!darwin
   1 !freebsd
   1 !dragonfly
   1 !darwin,!windows
   1 !darwin
   1 !cgo osusergo
   1 !cgo
   1 !arm64
   1 !appengine,linux freebsd darwin dragonfly netbsd openbsd
   1 !aix

The command I ran to list the unique GOOS combinations that trigger the build tag (ordered by frequency):

$ gron out.json | awk -F'"' '/file.content/{$1="";printf "%s%c", $0, 0}' | xargs -0 printf "%b" | awk -F"build " '/\+build/{print $2}' | xargs -n 1 | sort | uniq -c | sort -nr

Output

 303 !windows
 269 freebsd
 259 linux
 212 darwin
 186 openbsd
 183 netbsd
 164 dragonfly
 144 solaris
  53 aix
  42 hurd
  33 nacl
  17 js,wasm
   9 !android,linux
   7 !windows,!plan9
   7 !windows,!darwin
   6 !windows,!linux
   5 linux,!appengine
   5 !windows,!nacl,!plan9
   5 !appengine
   4 cgo
   4 !linux,!windows
   3 illumos
   3 !windows,!plan9,!linux,!openbsd
   3 !plan9,!windows
   3 !nofuse
   3 !linux,!darwin,!freebsd
   3 !js
   3 !android
   2 windows
   2 plan9
   2 linux,!baremetal
   2 linux,!android
   2 linux,!386
   2 js
   2 freebsd,!baremetal
   2 darwin,amd64
   2 darwin,386
   2 cgo,!osusergo
   2 cgo,!netgo
   2 android
   2 !plan9
   2 !nofuse,!openbsd,!netbsd
   2 !cgo
   1 osusergo
   1 openbsd,!appengine
   1 netbsd,!appengine
   1 linux,cgo
   1 js,!windows
   1 integration,benchmark,!windows
   1 freebsd,!appengine
   1 darwin,cgo
   1 darwin,!appengine
   1 !windows,!solaris,!plan9
   1 !windows,!plan9,!solaris,!linux
   1 !windows,!plan9,!solaris,!irix
   1 !windows,!plan9,!solaris,!aix
   1 !windows,!plan9,!solaris
   1 !windows,!plan9,!aix
   1 !windows,!openbsd,!netbsd,!nofuse
   1 !windows,!nofuse
   1 !windows,!freebsd,!dragonfly,!darwin
   1 !windows,!freebsd
   1 !windows,!darwin,!freebsd,!netbsd,!openbsd,!dragonfly,!solaris
   1 !windows,!darwin,!freebsd,!netbsd
   1 !windows,!android
   1 !solaris,!windows
   1 !solaris
   1 !openbsd
   1 !noplugin
   1 !netbsd
   1 !linux,!windows,!darwin
   1 !freebsd
   1 !dragonfly
   1 !darwin,!windows
   1 !darwin
   1 !arm64
   1 !appengine,linux
   1 !aix

What is interesting is the large majority of projects that take !windows to mean unix. Many exclude operating systems like hurd and aix. A few explicitly include non-unixes like js, nacl, plan9, and windows.

[seankhliao]

I scanned through latest version of all modules available in proxy.golang.org, resulting in 369323 build tag lines from approx 145000 modules, of which 115946 lines contained only platforms (excluding any lines with architectures, cgo, etc.)

unfiltered build tag lines (628k download, 8M decompressed): buildtags.txt.gz

the results have the build tags sorted for deduplication

   22103 windows
   20962 linux
   20055 !windows
    6752 !linux
    3682 darwin
    2770 !js
    2757 darwin dragonfly freebsd linux netbsd openbsd solaris
    2222 darwin dragonfly freebsd netbsd openbsd
    1698 darwin linux
    1689 darwin dragonfly freebsd linux netbsd openbsd
    1685 !plan9
    1632 solaris
    1567 js
    1566 aix darwin dragonfly freebsd linux netbsd openbsd solaris
    1506 freebsd linux
    1298 !android
    1199 darwin dragonfly freebsd linux nacl netbsd openbsd solaris
    1117 freebsd
    1100 darwin freebsd linux
     773 !darwin
     752 plan9
     671 !nacl
     616 aix
     580 openbsd
     565 darwin freebsd linux netbsd openbsd
     484 darwin linux windows
     478 !solaris
     478 darwin dragonfly freebsd linux nacl netbsd openbsd solaris windows
     469 android
     439 darwin dragonfly freebsd linux netbsd openbsd solaris windows
     436 dragonfly freebsd netbsd openbsd
     370 darwin freebsd netbsd openbsd
     347 aix darwin dragonfly freebsd linux netbsd openbsd solaris windows
     335 darwin freebsd linux openbsd
     292 darwin freebsd linux openbsd windows
     291 dragonfly freebsd linux netbsd openbsd
     285 freebsd netbsd openbsd
     277 darwin freebsd linux windows
     270 darwin freebsd
     266 aix darwin dragonfly freebsd linux netbsd openbsd
     265 !openbsd
     254 !freebsd
     254 nacl plan9 solaris
     242 !aix
     233 freebsd linux solaris
     225 darwin freebsd linux solaris
     224 nacl
     208 dragonfly freebsd linux
     180 !netbsd
     166 linux solaris
     162 aix linux
     155 !dragonfly
     154 nacl plan9
     152 nacl plan9 windows
     141 dragonfly freebsd linux netbsd openbsd solaris
     132 darwin dragonfly freebsd openbsd
     130 dragonfly
     129 dragonfly freebsd
     124 js nacl plan9
     120 darwin dragonfly freebsd linux netbsd openbsd plan9
     119 netbsd
     119 darwin dragonfly freebsd
     118 darwin freebsd solaris
     114 darwin freebsd netbsd
     109 freebsd openbsd
     106 aix darwin dragonfly freebsd linux nacl netbsd openbsd solaris
     104 plan9 windows
     102 darwin dragonfly freebsd linux nacl netbsd openbsd
     102 darwin freebsd linux openbsd solaris
     101 aix darwin dragonfly freebsd netbsd openbsd
     100 darwin dragonfly freebsd linux netbsd openbsd windows
      98 dragonfly netbsd openbsd
      92 darwin linux netbsd openbsd solaris
      88 darwin freebsd openbsd
      87 js nacl plan9 windows
      86 linux netbsd openbsd solaris
      78 dragonfly linux netbsd openbsd solaris
      77 darwin freebsd linux netbsd openbsd windows
      73 darwin linux openbsd windows
      72 freebsd linux netbsd openbsd
      69 darwin dragonfly freebsd netbsd
      69 dragonfly linux openbsd solaris
      68 dragonfly freebsd linux nacl netbsd openbsd solaris
      67 plan9 solaris windows
      66 darwin windows
      66 aix darwin dragonfly freebsd hurd linux netbsd openbsd solaris
      64 plan9 solaris
      63 aix darwin dragonfly freebsd linux netbsd openbsd plan9
      61 darwin freebsd linux netbsd openbsd solaris
      59 darwin linux solaris
      58 darwin dragonfly freebsd netbsd openbsd solaris
      58 netbsd openbsd
      55 darwin nacl netbsd openbsd plan9 solaris windows
      55 nacl solaris
      55 dragonfly freebsd netbsd
      54 dragonfly freebsd nacl netbsd openbsd solaris
      50 android dragonfly freebsd openbsd
      50 android linux solaris
      49 aix solaris
      48 freebsd netbsd
      48 linux windows
      46 darwin freebsd linux netbsd windows
      45 darwin dragonfly freebsd linux nacl netbsd openbsd plan9 solaris
      44 freebsd linux netbsd
      43 darwin dragonfly freebsd netbsd openbsd solaris windows
      43 darwin dragonfly linux netbsd openbsd solaris
      40 darwin freebsd linux netbsd
      40 darwin dragonfly netbsd
      39 !linux android
      38 darwin dragonfly freebsd nacl netbsd openbsd
      37 darwin dragonfly nacl netbsd openbsd solaris
      37 dragonfly freebsd linux netbsd
      37 aix darwin solaris
      36 dragonfly freebsd netbsd openbsd solaris
      35 darwin dragonfly freebsd illumos linux netbsd openbsd solaris
      35 darwin linux netbsd openbsd
      34 dragonfly freebsd windows
      34 !darwin linux
      32 !illumos
      30 dragonfly freebsd linux netbsd solaris
      30 zos
      29 freebsd linux openbsd solaris
      28 freebsd linux openbsd
      27 aix darwin freebsd linux netbsd openbsd solaris
      26 darwin dragonfly freebsd netbsd openbsd windows
      26 linux netbsd
      25 aix darwin dragonfly freebsd linux nacl netbsd openbsd solaris windows
      25 dragonfly freebsd solaris
      25 darwin dragonfly freebsd nacl netbsd openbsd solaris
      24 darwin linux openbsd
      24 darwin nacl netbsd openbsd
      23 aix dragonfly freebsd linux netbsd openbsd
      22 aix darwin dragonfly freebsd netbsd openbsd solaris
      21 dragonfly linux netbsd openbsd
      21 illumos
      20 aix darwin dragonfly freebsd js linux netbsd openbsd solaris
      20 aix darwin freebsd linux solaris
      20 dragonfly netbsd
      20 aix darwin dragonfly solaris
      20 freebsd solaris
      19 aix darwin dragonfly freebsd netbsd openbsd solaris windows
      19 aix freebsd linux netbsd
      19 aix freebsd netbsd openbsd
      19 aix openbsd
      19 aix darwin dragonfly freebsd openbsd
      19 aix linux netbsd
      19 hurd
      19 dragonfly openbsd solaris
      19 linux openbsd
      19 aix darwin dragonfly freebsd netbsd
      18 darwin nacl netbsd openbsd solaris
      18 linux netbsd openbsd
      18 dragonfly freebsd js netbsd openbsd
      17 !windows linux
      17 !plan9 !windows
      16 darwin linux netbsd solaris
      16 aix darwin dragonfly freebsd hurd linux netbsd openbsd solaris windows
      16 dragonfly plan9 solaris
      16 freebsd linux netbsd openbsd windows
      15 aix dragonfly linux openbsd solaris
      15 freebsd linux windows
      15 darwin dragonfly freebsd openbsd solaris
      14 darwin dragonfly freebsd linux openbsd
      13 freebsd linux netbsd solaris
      13 darwin dragonfly freebsd linux
      12 darwin dragonfly netbsd openbsd
      12 aix dragonfly freebsd linux netbsd openbsd solaris
      12 hurd linux
      12 !linux windows
      11 aix darwin dragonfly
      11 aix darwin netbsd openbsd plan9 solaris windows
      11 darwin freebsd openbsd solaris
      11 darwin dragonfly freebsd linux netbsd openbsd plan9 solaris
      11 aix linux solaris
      11 darwin dragonfly freebsd hurd linux netbsd openbsd solaris
      11 aix darwin dragonfly freebsd linux nacl netbsd openbsd plan9 solaris
      10 dragonfly linux nacl netbsd openbsd
      10 !hurd
      10 dragonfly linux nacl netbsd
      10 freebsd linux openbsd windows
      10 aix darwin dragonfly freebsd linux netbsd openbsd plan9 solaris
      10 dragonfly freebsd linux openbsd solaris
      10 android nacl plan9
      10 darwin nacl solaris
      10 !windows js
      10 dragonfly freebsd hurd linux netbsd openbsd
      10 freebsd linux netbsd openbsd solaris
       9 darwin freebsd linux netbsd openbsd plan9
       9 darwin freebsd linux nacl netbsd openbsd solaris
       9 dragonfly linux
       9 android darwin nacl netbsd plan9 windows
       9 openbsd solaris
       9 android freebsd openbsd
       9 darwin freebsd js linux windows
       8 darwin dragonfly freebsd linux openbsd solaris
       8 android darwin dragonfly freebsd linux netbsd openbsd solaris
       8 aix darwin nacl solaris
       8 netbsd openbsd plan9 solaris windows
       8 !linux darwin
       8 netbsd openbsd plan9 solaris
       7 darwin freebsd linux netbsd openbsd plan9 solaris zos
       7 android linux
       7 darwin dragonfly openbsd
       7 dragonfly nacl netbsd openbsd solaris
       7 dragonfly linux solaris
       7 !windows darwin linux
       7 aix darwin linux solaris
       7 freebsd netbsd openbsd windows
       7 aix darwin nacl netbsd openbsd plan9 solaris windows
       7 darwin dragonfly solaris
       7 darwin dragonfly freebsd solaris
       6 aix darwin dragonfly freebsd hurd netbsd openbsd solaris
       6 solaris windows
       6 !darwin !freebsd !linux !netbsd !openbsd windows
       6 android darwin freebsd linux netbsd openbsd
       6 dragonfly linux netbsd
       5 aix darwin dragonfly freebsd js linux nacl netbsd openbsd solaris
       5 dragonfly linux netbsd solaris
       5 !darwin !linux windows
       5 aix darwin dragonfly freebsd hurd linux nacl netbsd openbsd solaris
       5 dragonfly freebsd openbsd
       5 aix darwin freebsd netbsd openbsd
       5 linux nacl solaris
       5 darwin dragonfly freebsd js linux nacl netbsd openbsd solaris
       4 darwin dragonfly freebsd js linux netbsd openbsd solaris
       4 aix js nacl solaris
       4 darwin dragonfly freebsd hurd linux netbsd openbsd
       4 darwin openbsd
       4 !windows darwin freebsd linux
       4 aix darwin linux nacl solaris
       4 js linux nacl solaris
       4 dragonfly openbsd
       4 aix dragonfly freebsd linux netbsd solaris
       4 darwin dragonfly freebsd nacl netbsd openbsd plan9 solaris
       4 aix darwin dragonfly freebsd nacl netbsd openbsd plan9 solaris
       4 !darwin !freebsd !linux windows
       4 darwin nacl openbsd solaris
       3 aix android linux solaris
       3 android darwin dragonfly freebsd linux nacl netbsd openbsd solaris
       3 dragonfly js netbsd openbsd
       3 aix darwin dragonfly freebsd hurd linux nacl netbsd openbsd solaris windows
       3 darwin dragonfly nacl solaris
       3 aix android js nacl plan9 solaris windows
       3 !freebsd !netbsd !openbsd
       3 aix darwin dragonfly freebsd hurd js linux netbsd openbsd solaris
       2 netbsd plan9 windows
       2 freebsd hurd netbsd openbsd
       2 darwin linux netbsd
       2 !linux darwin dragonfly freebsd netbsd openbsd
       2 aix darwin freebsd js linux nacl netbsd openbsd solaris
       2 aix dragonfly freebsd netbsd openbsd solaris
       2 js plan9
       2 aix dragonfly freebsd linux nacl netbsd openbsd solaris
       2 !linux !windows
       2 aix darwin dragonfly freebsd hurd netbsd openbsd
       2 android darwin freebsd linux openbsd
       2 darwin freebsd linux netbsd openbsd plan9 solaris
       2 android darwin linux windows
       2 darwin freebsd illumos linux openbsd solaris
       2 android windows
       2 aix dragonfly hurd linux openbsd solaris
       2 darwin netbsd openbsd
       2 darwin dragonfly freebsd hurd netbsd openbsd
       2 darwin dragonfly nacl openbsd solaris
       2 darwin dragonfly freebsd linux nacl netbsd openbsd windows
       2 freebsd linux nacl netbsd
       2 aix darwin dragonfly freebsd hurd linux netbsd openbsd plan9
       2 netbsd openbsd solaris
       2 aix darwin dragonfly freebsd hurd linux netbsd openbsd
       2 freebsd windows
       2 darwin dragonfly openbsd solaris
       2 dragonfly js netbsd openbsd solaris
       2 aix freebsd hurd linux netbsd
       1 !linux darwin freebsd netbsd openbsd windows
       1 !solaris darwin dragonfly freebsd linux netbsd openbsd plan9
       1 aix darwin freebsd hurd linux netbsd openbsd solaris
       1 freebsd illumos netbsd solaris
       1 aix darwin dragonfly freebsd hurd linux netbsd openbsd plan9 solaris
       1 darwin freebsd linux netbsd solaris
       1 openbsd windows
       1 darwin dragonfly freebsd linux nacl netbsd openbsd plan9 solaris windows
       1 dragonfly linux openbsd
       1 aix dragonfly freebsd hurd linux netbsd openbsd solaris
       1 aix darwin dragonfly freebsd hurd linux nacl netbsd openbsd plan9 solaris
       1 darwin solaris windows
       1 dragonfly freebsd linux openbsd
       1 aix darwin dragonfly freebsd hurd js linux nacl netbsd openbsd solaris
       1 !linux !windows darwin
       1 darwin netbsd
       1 !darwin !windows linux
       1 js plan9 windows
       1 dragonfly freebsd linux netbsd openbsd plan9 solaris
       1 linux plan9 windows
       1 aix darwin hurd netbsd openbsd plan9 solaris windows
       1 aix darwin hurd nacl netbsd openbsd plan9 solaris windows
       1 freebsd hurd linux netbsd openbsd solaris
       1 !windows darwin freebsd linux netbsd openbsd
       1 android darwin freebsd linux netbsd openbsd solaris
       1 android nacl plan9 windows
       1 android darwin linux
       1 darwin dragonfly freebsd illumos netbsd openbsd solaris
       1 darwin dragonfly netbsd openbsd solaris
       1 aix hurd linux solaris
       1 illumos solaris
       1 !darwin !linux

source code: messy

[smasher164]

@seankhliao Is that list a collection of all build tags, or only those for "unix"? If it's the former, it'd be interesting to know what fraction of them would be affected by the addition of a unix build tag.

[seankhliao]

@smasher164 the attached file has all the build tags. The list in my comment filters it to count the lines with only platforms.

Given the current list, I don't think any of the lines exactly matches all the unixes as listed by @rsc (android appears really unpopular), though the correctness of the build tag lines in the wild can be debated (as people may forget about the less common platforms).

[rsc]

Thanks for the data, @seankhliao and @smasher164. It does look like having unix would have avoided a lot of entropy in the current build lines. Given that possible reduction in entropy along with possibly simplifying Unix-like ports (as discussed in #37503), it sounds like we should probably adopt this proposal.

Quoting my previous message:

For sake of discussion, unix would be all the operating systems traditionally recognized as Unix (aix, android, darwin, dragonfly, freebsd, hurd, illumos, linux, netbsd, openbsd, solaris; NOT js, nacl, plan9, windows, zos). It is of course ironic that GNU Hurd is unix by this definition.

Does anyone want to argue against adopting this proposal?

[rsc]

Based on the discussion above, this seems like a likely accept. (See previous comment for exact definition of "unix".)

[rsc]

No change in consensus, so accepted.

[bitfield]

Has this been implemented? If so, I haven't found any documentation about it. Adding an explicit //go:build unix to a file excludes it from the build on macOS, so it doesn't seem to be meaningful to Go.

[mvdan]

@bitfield this proposal is accepted but still open, meaning it hasn't been implemented. It doesn't appear like anyone is actively working on it, so it's up for grabs if someone has the time to work it out :)

[ianlancetaylor]

I took a look at the _unix.go files in the standard library. There are several such files that have a //go:build line that matches non-Unix systems. This is not a big deal, in that we can rename the files (I'm thinking of _unixy.go) and carry on. But if the same pattern happens in packages outside the standard library, then changing the treatment of _unix.go files will break those packages.

It may be safer to define the unix build tag in go:build lines, but to not recognize it in file names. There are of course other similar build tags, such as cgo or gc or go1.18. That seems like a conservative approach to me. Does anybody think that it would be a mistake?

Here are some examples from the standard library (this is not a complete list). If we recognize _unix.go files as we do _GOOS.go files, each of these files (and more) will have to be renamed to make the build succeed.

time/sys_unix.go: //go:build aix || darwin || dragonfly || freebsd || (js && wasm) || linux || netbsd || openbsd || solaris cmd/go/internal/base/signal_unix.go: //go:build aix || darwin || dragonfly || freebsd || js || linux || netbsd || openbsd || solaris cmd/go/stop_unix_test.go: //go:build aix || darwin || dragonfly || freebsd || (js && wasm) || linux || netbsd || openbsd || solaris crypto/rand/rand_unix.go://go:build aix || darwin || dragonfly || freebsd || linux || netbsd || openbsd || plan9 || solaris path/filepath/path_unix.go: //go:build aix || darwin || dragonfly || freebsd || (js && wasm) || linux || netbsd || openbsd || solaris golang.org/x/sys/unix/sockcmsg_unix.go: //go:build aix || darwin || dragonfly || freebsd || linux || netbsd || openbsd || solaris || zos syscall/env_unix.go: //go:build aix || darwin || dragonfly || freebsd || (js && wasm) || linux || netbsd || openbsd || solaris || plan9 net/error_unix.go: //go:build aix || darwin || dragonfly || freebsd || js || linux || netbsd || openbsd || solaris os/exec/exec_unix.go: //go:build !plan9 && !windows os/signal/signal_unix.go: //go:build aix || darwin || dragonfly || freebsd || (js && wasm) || linux || netbsd || openbsd || solaris || windows

[gopherbot]

Change https://go.dev/cl/389935 mentions this issue: all: use new "unix" build tag where appropriate

[gopherbot]

Change https://go.dev/cl/389934 mentions this issue: go/build: recognize "unix" build tag

[ianlancetaylor]

The CL https://go.dev/cl/389934 implements "unix" as a build tag, but only in go:build lines, not in file names.

The CL https://go.dev/cl/389935 shows how the standard library can take advantage of the new build tag.

[rsc]

Let's move discussion to #51572.

[gopherbot]

Change https://go.dev/cl/426296 mentions this issue: cmd/go/internal/imports: recognize "unix" build tag

[gopherbot]

Change https://go.dev/cl/426814 mentions this issue: [release-branch.go1.19] cmd/go/internal/imports: recognize "unix" build tag

[gopherbot]

Change https://go.dev/cl/437235 mentions this issue: all: use "unix" build tag where appropriate