search

golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
124.32k stars 17.7k forks source link

build: add GOOS=illumos #20603

Closed sean- closed 5 years ago

sean- commented 7 years ago

Disambiguating solaris vs illumos

SunOS/Solaris has a storied and complicated history. In [2010] Illumos forked from OpenSolaris and has continued its life in the open (, and now spent the majority of its life, as Illumos (not Solaris).

The solaris build tag is "mostly" compatible with Illumos and Illumos-based distributions (e.g. SmartOS, Nexenta, Open Indiana, Delphix, etc), however Illumos has diverged significantly from Solaris. In order to detect and support Illumos-native functionality, I propose:

  1. illumos be added as a new GOOS build tag
  2. The illumos build tag be distinct from solaris.

We considered extending the life of the solaris build target to include the illumos target for the period of one release but decided against this because it would taint community code with:

// +build !solaris
// +build illumos

that would need to be cleaned up at the end of the transition period. Backwards compatibility for the sake of backwards compatibility isn't something we're interested in maintaining.

Semi-related: it would be nice if there was a way of specifying and targeting distributions at build time. cgo on Linux and alpine vs glibc comes to mind as another area that would benefit from a distribution-specific build target.

Smithx10 commented 5 years ago

@bradfitz Does illumos require someone to maintain the Oracle Solaris port? When @jclulow has the SmartOS builder running ( https://twitter.com/jmclulow/status/1117305055755919360 ) and a community member takes the roll of maintainer, I'd imagine it would be desirable for the tag to be completely independent from the Oracle Solaris lifecycle. What do you think?

bradfitz commented 5 years ago

I'm not going to ask that an illumos contributor maintains the Oracle Solaris port (and its builder).

But it'd be a sad situation if we split the current GOOS in two and one of the halves dies due to neglect.

Does anybody in this bug care about the GOOS=solaris port existing if GOOS=illumos exists? I'm not sure where to find a GOOS=solaris maintainer. My contacts at Oracle dried up and nobody replied to my golang-dev post.

jclulow commented 5 years ago

@bradfitz Speaking as an illumos community member, I'm chiefly interested in a GOOS=illumos that allows us to provide support for things like flock() that we've perhaps done differently from Solaris. The possibility of having GOOS=illumos imply GOOS=solaris seems very attractive, as it would presumably allow us to make immediate forward progress on the differences without throwing out all the software currently working by using the solaris tag today.

I agree that it's something of a conundrum. I don't think you can even get Solaris 11 for free, at least not with security updates and bug fixes beyond what is in the major release ISO. I know a few people that work at Oracle and I will ask if they know of anybody inside who might be interested, but I'm not sure there will be anyone.

If it is eventually decided that the GOOS=solaris tag isn't viable anymore due to lack of maintainers, would we still be able to have GOOS=illumos imply it in the future? I'm not yet deeply familiar with how the overlapping/implying tag stuff works.

bradfitz commented 5 years ago

If it is eventually decided that the GOOS=solaris tag isn't viable anymore due to lack of maintainers, would we still be able to have GOOS=illumos imply it in the future? I'm not yet deeply familiar with how the overlapping/implying tag stuff works.

Yeah, if GOOS=solaris does get removed, we could keep "solaris" as an additional implicit build tag when GOOS=illumos.

gopherbot commented 5 years ago

Change https://golang.org/cl/174083 mentions this issue: all: add new GOOS=illumos, split out of GOOS=solaris

bradfitz commented 5 years ago

I sent https://go-review.googlesource.com/c/go/+/174083/ as an example CL of what would be required to do this. Anybody want to pick it up, test it, and finish it?

You have a few days before Go 1.13 goes into freeze.

jclulow commented 5 years ago

@bradfitz Hi Brad! I can take a look at it, though that timeline is obviously a little tight as I don't have a strong background on all of the infrastructure here. When is the exact date of the freeze, and if we miss this round does that mean we're effectively waiting for Go 1.14? (When is that?)

As an aside, I have a rough prototype of a Virtio SCSI driver and some other changes required to get illumos running under GCE directly. I'm learning a bit more about how to parcel up and share a useful GCE image that others can consume.

bradfitz commented 5 years ago

Freezing May 1st. Tree opens again 3 months after that. https://golang.org/wiki/Go-Release-Cycle

jclulow commented 5 years ago

Thanks!

jclulow commented 5 years ago

@bradfitz I spent some time on this yesterday. I took your initial CL (thanks for the push in the right direction!) and have added in a bunch of extra changes, in parts using the android/linux split as a guide.

I have been able to cross build an illumos/amd64 toolchain using a stock solaris/amd64 1.12.1 toolchain I had available already. Using this cross built toolchain, I was then able to bootstrap a native build of the illumos/amd64 toolchain. All of the tests appear to pass on the final toolchain in this sequence.

I've enrolled my work gsuite account in the Gerrit instance, and I've kicked off the CLA signing process. My boss has signed it this morning, but I'm not sure how long it will take to show up in the system -- I can't see it yet.

Will I be able to adopt your existing CL and push an updated change to that, or should I start a new CL with the git codereview tool?

jclulow commented 5 years ago

@bradfitz I'd really like to get this in before the gate closes on May 1st. I've pushed the change that I have so far (which I feel is ready to go) to a branch on Github: https://github.com/joyent/go/commit/9ee73d27649568275117bc48764a4679bbf7fbf2

I believe I've done all the right things with the HTTP cookies file and Gerrit, but I still can't appear to submit a CL:

$ git codereview pending -c
illumos 2b8cbc3..9ee73d2764 (current branch, 1 behind)
+ 9ee73d2764
    all: add new GOOS=illumos, split out of GOOS=solaris

    Like GOOS=android which implies the "linux" build tag, GOOS=illumos
    implies the "solaris" build tag. This lets the existing ecosystem of
    packages still work on illumos, but still permits packages to start
    differentiating between solaris and illumos.

    Fixes #20603

    Change-Id: I8f4eabf1a66060538dca15d7658c1fbc6c826622

    Files in this change:
        misc/cgo/test/sigaltstack.go
        misc/cgo/testcarchive/carchive_test.go
        misc/cgo/testcshared/cshared_test.go
        src/cmd/dist/build.go
        src/cmd/dist/main.go
        src/cmd/go/go_test.go
        src/cmd/go/internal/bug/bug.go
        src/cmd/go/internal/imports/build.go
        src/cmd/go/internal/lockedfile/internal/filelock/filelock_test.go
        src/cmd/go/internal/work/buildid.go
        src/cmd/go/internal/work/exec.go
        src/cmd/go/internal/work/init.go
        src/cmd/internal/objabi/head.go
        src/cmd/link/internal/ld/dwarf_test.go
        src/cmd/vendor/golang.org/x/sys/unix/sockcmsg_unix.go
        src/debug/gosym/pclntab_test.go
        src/go/build/build.go
        src/go/build/doc.go
        src/go/build/syslist.go
        src/net/error_test.go
        src/net/fd_unix.go
        src/net/interface_test.go
        src/net/listen_test.go
        src/net/tcpsock_test.go
        src/os/exec/exec_test.go
        src/os/os_test.go
        src/os/removeall_test.go
        src/os/user/user_test.go
        src/runtime/asm_amd64.s
        src/runtime/cgocall.go
        src/runtime/crash_unix_test.go
        src/runtime/internal/sys/gengoos.go
        src/runtime/internal/sys/zgoos_aix.go
        src/runtime/internal/sys/zgoos_android.go
        src/runtime/internal/sys/zgoos_darwin.go
        src/runtime/internal/sys/zgoos_dragonfly.go
        src/runtime/internal/sys/zgoos_freebsd.go
        src/runtime/internal/sys/zgoos_hurd.go
        src/runtime/internal/sys/zgoos_illumos.go
        src/runtime/internal/sys/zgoos_js.go
        src/runtime/internal/sys/zgoos_linux.go
        src/runtime/internal/sys/zgoos_nacl.go
        src/runtime/internal/sys/zgoos_netbsd.go
        src/runtime/internal/sys/zgoos_openbsd.go
        src/runtime/internal/sys/zgoos_plan9.go
        src/runtime/internal/sys/zgoos_solaris.go
        src/runtime/internal/sys/zgoos_windows.go
        src/runtime/internal/sys/zgoos_zos.go
        src/runtime/mem_bsd.go
        src/runtime/netpoll.go
        src/runtime/pprof/pprof_test.go
        src/runtime/proc.go
        src/runtime/rt0_illumos_amd64.s
        src/syscall/exec_unix.go
        src/syscall/sockcmsg_unix.go
        src/syscall/syscall_unix.go
$ git codereview mail
fatal: remote error: PERMISSION_DENIED The caller does not have permission
(running: git push -q origin HEAD:refs/for/master)
/home/jclulow/bin/git-codereview: exit status 128

We're having a bit of trouble with the CLA process. It seems a departed Joyent employee might have signed a CLA in the past, and we're struggling through the process of adopting/reactivating it at the moment. Would that prevent my being able to submit to Gerrit?

Smithx10 commented 5 years ago

@jclulow Whoa! This is great news! Hopefully @bradfitz can help clear up the CLA before the deadline and Illumos may finally have it's own tag. Definitely a win for both communities. Can't wait!

bradfitz commented 5 years ago

We're having a bit of trouble with the CLA process. It seems a departed Joyent employee might have signed a CLA in the past, and we're struggling through the process of adopting/reactivating it at the moment. Would that prevent my being able to submit to Gerrit?

There are two ways to submit code to Go: using Gerrit, or using GitHub PRs. Both require the same CLA process. We can't accept code without a CLA.

It looks like you need to join the go-contributors@$YOUR_COMPANY_DOMAIN mailing list.

jclulow commented 5 years ago

Thanks, Brad. I'll follow up with our internal IT folks tomorrow to see if we have one of those.

bradfitz commented 5 years ago

My guess is that it's a Google Group, so the URL would be:

https://groups.google.com/a/joyent.com/d/forum/go-contributors or https://groups.google.com/a/joyent.com/forum/#!forum/go-contributors

... which I think are identical.

gopherbot commented 5 years ago

Change https://golang.org/cl/174457 mentions this issue: all: add new GOOS=illumos, split out of GOOS=solaris

jclulow commented 5 years ago

@bradfitz, you're a steely-eyed missile man!

I've uploaded a CL: https://go-review.googlesource.com/c/go/+/174457

gopherbot commented 5 years ago

Change https://golang.org/cl/174958 mentions this issue: unix: add illumos case

gopherbot commented 5 years ago

Change https://golang.org/cl/179997 mentions this issue: cmd/go/internal/imports: match solaris files for illumos

bcmills commented 5 years ago

The Illumos builder was removed due to the upcoming Joyent Public Cloud turndown.

If folks want to maintain support for the illumos port, it needs a builder. See #15581.

jclulow commented 5 years ago

I'll take a look!