proposal: all: add GOOS=cosmo port for Cosmopolitan Libc

[borancar]

Go is slowly becoming the de-facto lingua franca of DevOps - to the point of replacing shell scripting (see Cloudflare Blog/Using Go as a scripting language in Linux). The next great iteration would be a compile once run anywhere type of such scripting (as possible as it can be with external packaging involved). There's already a PoC libc and executable format that can run anywhere (see Cosmopolitan libc and APE). The author of Cosmpolitan and APE has indicated a willingness to collaborate and support Go running on Cosmopolitan and producing APE binaries.

[ianlancetaylor]

Without looking too hard, I think this would effectively mean porting Go to a new GOOS value. See https://go.dev/wiki/PortingPolicy.

It would be interesting to hear how much work this would be. To be clear, it is extremely unlikely that anybody on the core Go team would work on this. But if someone is interested in working on it, and can satisfy the porting policy, we could consider adding the port.

[dmitshur]

CC @golang/release, @aclements as FYI.

[rsc]

It does sound like this would be a new operating system, which somehow is the union of all existing operating systems. How does Cosmopolitan libc measure up against the requirements in #53383?

And what would the changes look like in the runtime? I am a little scared of the "union of all systems" aspect of this. If Cosmopolitan libc is taking care of all this, where would Go find the libc object to link in?

[rsc]

This proposal has been added to the active column of the proposals project and will now be reviewed at the weekly proposal review meetings. — rsc for the proposal review group

[jart]

Cosmopolitan Libc author here. Thank you for considering this proposal. Our project would love the opportunity to support the Go community.

[jart]

To answer @rsc's questions:

And what would the changes look like in the runtime?

Try taking an existing libc integration (e.g. go/src/runtime/sys_openbsd2.go) and using that with cosmopolitan.h and cosmopolitan.a. You need to use ld.bfd to link your binaries unless you generate the APE executable format on your own. Cosmopolitan assumes your entrypoint will be main(). Everything is System V ABI. We do backtraces using %rbp. We log function calls using mcount. If you want TLS on all operating systems then you need to devote both %fs and %gs since the latter is dynamically rewritten on Windows and Apple platforms. If Go has register requirements that aren't SysV you can tune the build using flags like -ffixed-rbx. Any place where you've hardcoded system-related magic numbers (e.g. EPERM) would need to be modified to fetch it from an extern errno_t instead.

I am a little scared of the "union of all systems" aspect of this.

Cosmopolitan is the union of all operating systems in the sense that jQuery was the union of all web browsers. The only big thing we changed was we made POSIX defines symbolic. For example, #define EPERM 1 is now extern errno_t EPERM;. That let us abstract most of the portability issues between UNIX variants. Then we wrote system call veneers that polyfill behavior differences between platforms like Windows. https://justine.lol/ape.html

If Cosmopolitan libc is taking care of all this, where would Go find the libc object to link in?

Consider using one of our pinned "amalgamation" releases. We put everything in a giant static archive. You need cosmopolitan.a, crt.o, ape-no-modify-self.o, ape.lds, and cosmopolitan.h which are all in the zip file.

• https://github.com/jart/cosmopolitan/releases/tag/2.0 (latest release)
• https://justine.lol/cosmopolitan/cosmopolitan-amalgamation-2.0.zip (official download link)
• https://worker.jart.workers.dev/cosmopolitan-amalgamation-2.0.zip (please use this free egress mirror for your CIs)

Actual or expected user base. If there are relatively few users, significant effort to maintain a port may not be worthwhile.

Our Actually Portable Executable web server (redbean) is the no. 3 most upvoted Show HN thread in Hacker News history. See https://bestofshowhn.com/ and https://redbean.dev/

[rsc]

Given that https://github.com/golang/go/issues/53383 has been accepted (clarifying our porting policy), I think that a cosmo libc port would be treated like any other secondary operating system port, and subject to the requirements there. If someone wants to start the port, you can assume GOOS=cosmo, although I don't think we'll reserve that until a port appears. Retitled issue to make clear that this is a new port, not a runtime option.

There will be interesting translations necessary. The SysV ABI doesn't matter much, since it only affects C code we call on the m stacks. The variable values for things like EPERM mean that we'll have to translate to actual constants of our own choosing to preserve the constant-ness of syscall.EPERM etc on the Go side. And I assume we'd require invoking the external linker to make a binary.

It's not something the Go team is going to implement ourselves, but a new port is welcome subject to the considerations in #53383, just as we would any other port.

Putting this proposal on hold until someone tries to write a port.

[rsc]

Placed on hold. — rsc for the proposal review group

[cristaloleg]

@rsc looks like #53383 is effectively done (and even its continuation #56679), should this proposal be un-holded?

[ianlancetaylor]

@cristaloleg This proposal is on hold until somebody writes a GOOS=cosmo port. Does that exist somewhere? Thanks.

[Pikamander2]

This issue was mentioned in a Hacker News thread about GCC patching for greater portability.

https://news.ycombinator.com/item?id=36714967

[mauri870]

I wonder if it would be possible to use cosmocc to build with cgo, currently it hangs forever in runtime/cgo

$ CGO_ENABLED=1 CC=cosmocc go build -v --ldflags '-linkmode external -extldflags=-static' main.go
runtime/cgo

[mauri870]

Cosmopolitan 3.0 just released

https://justine.lol/cosmo3/