search

caddyserver / caddy

Fast and extensible multi-platform HTTP/1-2-3 web server with automatic HTTPS
https://caddyserver.com
Apache License 2.0
58.01k stars 4.03k forks source link

Caddy Build Fails for Solaris/Illumos and Plan9 #3615

Closed mohammed90 closed 6 months ago

mohammed90 commented 4 years ago

Initially reported by @Toasterson

The main issue is that badger@1.5.3 uses golangs internal syscall library and that does not support many operatingsystems notably openbsd, netbsd, and illumos. Badger uses a constant for MADVISE and that only came into the go for these Unixes after the internal syscall library got frozen. Right now the internal syscall library only really works for linux and every software that wants to support something else than linux and uses the syscall library directly must use x/sys/unix instead. I've personally changed quite a few direct dependencies and pushed people to use x/sys/unix instead of syscall so that today this is mostly happening due the dependency chain. I could not upgrade badger to 1.6.1 as there where api changes but nosql bump worked. So until that is resolved caddy cannot be built on quite a few unix versions I suppose. I know it won't build for illumos right now. Are you monitoring cross-builds?

Also here the exact error for documentation

# github.com/dgraph-io/badger/y
../../../pkg/mod/github.com/dgraph-io/badger@v1.5.3/y/mmap_unix.go:57:30: undefined: syscall.SYS_MADVISE

Originally posted by @Toasterson in https://github.com/caddyserver/caddy/pull/3602#issuecomment-663876698

Build errors:

Known as of v2.1.1, but issue could be extending back in history. Exact version/commit is pretty much irrelevant.

mholt commented 4 years ago

I have opened issues upstream:

I've also removed those platforms from our download page for the time being. Too bad. :(

mohammed90 commented 4 years ago

I have opened issues upstream:

* https://discuss.dgraph.io/t/build-failure-for-goos-solaris-illumos-plan9/9145?u=mholt

* [etcd-io/bbolt#231](https://github.com/etcd-io/bbolt/issues/231)

* [chzyer/readline#188](https://github.com/chzyer/readline/issues/188)

I've also removed those platforms from our download page for the time being. Too bad. :(

There's one item in this stream 🙂 

# github.com/caddyserver/caddy/v2/cmd
../proc_posix.go:28:9: undefined: syscall.Kill

It's a thoughie because there's no equivalent to syscall.Kill on Plan9. If all upstream issues are resolved, we might need to resort to shell command to issue kill command.

mholt commented 4 years ago

There's one item in this stream 🙂

Yeah, but that one won't be too hard -- like you said, a shell command or something: https://en.wikipedia.org/wiki/Kill_(command)#Plan_9_from_Bell_Labs kill caddy | rc

Tbh I'm skeptical that the upstream ones will be resolved though.

mholt commented 4 years ago

Dgraph fixing builds for Plan9; looking into Solaris: https://discuss.dgraph.io/t/build-failure-for-goos-solaris-illumos-plan9/9145/3?u=mholt

lispstudent commented 3 years ago

Joyent's pkgsrc somehow manages to compile caddy on illumos, but it is with no packages.

francislavoie commented 3 years ago

Looks like it's working everywhere except plan9 now! https://github.com/caddyserver/caddy/actions/runs/1152172382

Errors I see from trying to build it locally:

# go.etcd.io/bbolt
..\..\..\..\go\pkg\mod\go.etcd.io\bbolt@v1.3.5\db.go:223:12: undefined: flock
..\..\..\..\go\pkg\mod\go.etcd.io\bbolt@v1.3.5\db.go:360:12: undefined: mmap
..\..\..\..\go\pkg\mod\go.etcd.io\bbolt@v1.3.5\db.go:382:12: undefined: munmap
..\..\..\..\go\pkg\mod\go.etcd.io\bbolt@v1.3.5\db.go:462:12: undefined: fdatasync
..\..\..\..\go\pkg\mod\go.etcd.io\bbolt@v1.3.5\db.go:507:14: undefined: funlock
..\..\..\..\go\pkg\mod\go.etcd.io\bbolt@v1.3.5\db.go:867:37: undefined: fdatasync
..\..\..\..\go\pkg\mod\go.etcd.io\bbolt@v1.3.5\tx.go:559:13: undefined: fdatasync
..\..\..\..\go\pkg\mod\go.etcd.io\bbolt@v1.3.5\tx.go:596:13: undefined: fdatasync
# github.com/chzyer/readline
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\operation.go:234:4: undefined: ClearScreen
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\readline.go:129:20: undefined: GetScreenWidth
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\readline.go:132:22: undefined: DefaultIsTerminal
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\readline.go:142:26: undefined: DefaultOnWidthChanged
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\remote.go:324:2: undefined: DefaultOnWidthChanged
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\remote.go:346:17: undefined: GetScreenWidth
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\remote.go:362:16: undefined: DefaultIsTerminal
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\terminal.go:51:2: undefined: SuspendMe
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\utils.go:81:29: undefined: State
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\utils.go:241:9: undefined: State
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\terminal.go:51:2: too many errors
# github.com/dgraph-io/badger/v2/y
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\file_dsync.go:24:21: undefined: unix.O_DSYNC
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\mmap_unix.go:30:11: undefined: unix.PROT_READ
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\mmap_unix.go:32:12: undefined: unix.PROT_WRITE
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\mmap_unix.go:34:9: undefined: unix.Mmap
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\mmap_unix.go:34:54: undefined: unix.MAP_SHARED
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\mmap_unix.go:39:9: undefined: unix.Munmap
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\mmap_unix.go:46:11: undefined: unix.MADV_NORMAL
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\mmap_unix.go:48:11: undefined: unix.MADV_RANDOM
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\mmap_unix.go:50:9: undefined: unix.Madvise

Asked badger if they can backport their fixes for plan9 to their v2 branch: https://github.com/dgraph-io/badger/pull/1451#issuecomment-903453493

github.com\chzyer\readline Looks essentially abandoned so we shouldn't hold our breath on that one.

mholt commented 2 years ago

Getting closer, sort of. Still builds for solaris OK, latest attempt at plan9 is now failing with:

$ GOOS=plan9 go build
# go.etcd.io/bbolt
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/db.go:230:12: undefined: flock
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/db.go:375:12: undefined: mmap
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/db.go:404:12: undefined: munmap
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/db.go:508:12: undefined: fdatasync
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/db.go:554:14: undefined: funlock
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/db.go:914:37: undefined: fdatasync
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/mlock_unix.go:14:17: undefined: unix.Mlock
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/mlock_unix.go:32:17: undefined: unix.Munlock
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/tx.go:558:13: undefined: fdatasync
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/tx.go:595:13: undefined: fdatasync
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/tx.go:595:13: too many errors
# github.com/chzyer/readline
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/utils.go:81:29: undefined: State
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/utils.go:241:9: undefined: State
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/operation.go:234:4: undefined: ClearScreen
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/readline.go:129:20: undefined: GetScreenWidth
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/readline.go:132:22: undefined: DefaultIsTerminal
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/readline.go:142:26: undefined: DefaultOnWidthChanged
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/remote.go:324:2: undefined: DefaultOnWidthChanged
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/remote.go:346:17: undefined: GetScreenWidth
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/remote.go:362:16: undefined: DefaultIsTerminal
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/terminal.go:51:2: undefined: SuspendMe
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/terminal.go:51:2: too many errors
# github.com/tailscale/tscert/internal/paths
/home/matt/go/pkg/mod/github.com/tailscale/tscert@v0.0.0-20220316030059-54bbcb9f74e2/internal/paths/paths_unix.go:44:15: undefined: unix.Access
/home/matt/go/pkg/mod/github.com/tailscale/tscert@v0.0.0-20220316030059-54bbcb9f74e2/internal/paths/paths_unix.go:44:32: undefined: unix.O_RDWR
mholt commented 6 months ago

Welp, still the same result 2 years later. I don't think there's any hope of getting Caddy to build on Plan9 until the listed dependencies also support it. We could be more proactive about getting this done, but I think we'll only prioritize this if there's a real need and/or sponsorship to do so.