search

machine-drivers / docker-machine-driver-xhyve

docker-machine/minikube/minishift driver plugin for xhyve/hyperkit (native macOS hypervisor.framework)
https://godoc.org/github.com/machine-drivers/docker-machine-driver-xhyve
BSD 3-Clause "New" or "Revised" License
888 stars 74 forks source link

Building doesn't make lib9p #98

Closed PerilousApricot closed 7 years ago

PerilousApricot commented 8 years ago

I'm working on updating the homebrew configuration to be able to build the HEAD https://github.com/Homebrew/homebrew/pull/49467 . I'm not a go developer, so I may be missing something, but it appears that nothing makes the vendor/build/* directories, and go build doesn't set the proper environment variables for cgo to find the lib9p build.

What is the "recommended" way to build from source? The README has instructions to fetch precompiled binaries from either brew or via go get, and my (uninformed) attempts to "go deps" or "go build" to trigger lib9p's compilation didn't work,, which led to my admittedly gross little hack.

zchee commented 8 years ago

@PerilousApricot Ah sorry, I do not yet update README :( and, Thanks fix formula pull request for homebrew.

I will post recommended build way later. Please wait for a little.

zchee commented 8 years ago

@PerilousApricot Currently, HEAD build needs lib9p.a static library. https://github.com/zchee/docker-machine-driver-xhyve/blob/master/Makefile#L64

So, can not successful build on go build one command. But, users can build one-step that command is make.

make command result is,

==> Build docker-machine-driver-xhyve...
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/mcnflag
github.com/zchee/docker-machine-driver-xhyve/vendor/golang.org/x/crypto/curve25519
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/docker/pkg/term
github.com/zchee/docker-machine-driver-xhyve/vendor/golang.org/x/crypto/ssh/terminal
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/log
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/state
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/version
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/hooklift/xhyve
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/johanneswuerbach/nfsexports
github.com/zchee/docker-machine-driver-xhyve/vmnet
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/mcnutils
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/drivers/plugin/localbinary
github.com/zchee/docker-machine-driver-xhyve/b2d
github.com/zchee/docker-machine-driver-xhyve/vendor/golang.org/x/crypto/ssh
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/ssh
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/drivers
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/drivers/rpc
github.com/zchee/docker-machine-driver-xhyve/xhyve
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/drivers/plugin
github.com/zchee/docker-machine-driver-xhyve

FYI, internal of Makefile build way is,

https://github.com/zchee/docker-machine-driver-xhyve/blob/master/mk%2Flib9p.mk#L33-L35

${GIT_CMD} submodule update --init
mkdir -p ${LIB9P_BUILD_DIR} ${LIB9P_BUILD_DIR}/sbuf ${LIB9P_BUILD_DIR}/transport ${LIB9P_BUILD_DIR}/backend

https://github.com/zchee/docker-machine-driver-xhyve/blob/master/mk%2Flib9p.mk#L37-L41

$(CC) $(LIB9P_CFLAGS) -c $< -o $@
$(LIBTOOL) -static $^ -o $@

https://github.com/zchee/docker-machine-driver-xhyve/blob/master/Makefile#L63-L64 https://github.com/zchee/docker-machine-driver-xhyve/blob/master/Makefile#L117-L121

$ CGO_CFLAGS=${CGO_CFLAGS} CGO_LDFLAGS=${CGO_LDFLAGS} ${GO_BUILD} -ldflags "$(GO_LDFLAGS)" ${GO_GCFLAGS} ${TOP_PACKAGE_DIR}/${PACKAGE}

In fact,

$ CGO_CFLAGS='-I/Users/zchee/go/src/github.com/zchee/docker-machine-driver-xhyve/vendor/lib9p' CGO_LDFLAGS='/Users/zchee/go/src/github.com/zchee/docker-machine-driver-xhyve/vendor/build/lib9p/lib9p.a -L/Users/zchee/go/src/github.com/zchee/docker-machine-driver-xhyve/vendor/lib9p' go build -v -o bin/docker-machine-driver-xhyve -ldflags  -w -s -X github.com/zchee/docker-machine-driver-xhyve/xhyve.GitCommit=68de456  github.com/zchee/docker-machine-driver-xhyve

If you have other questions, please feel free to comment.

zchee commented 8 years ago

@PerilousApricot Oh, sorry I missed. Could not buildmake one command until a while ago.

Fixed. See https://github.com/zchee/docker-machine-driver-xhyve/commit/0cf6631e3da723b76790f99e36c2ba01b0e9363b

make command result,

$ make
git submodule update --init
mkdir -p vendor/build/lib9p vendor/build/lib9p/sbuf vendor/build/lib9p/transport vendor/build/lib9p/backend
clang -Weverything -Wno-padded -Wno-gnu-zero-variadic-macro-arguments -Wno-format-nonliteral -Werror -g -O0 -c vendor/lib9p/pack.c -o vendor/build/lib9p/pack.o
clang -Weverything -Wno-padded -Wno-gnu-zero-variadic-macro-arguments -Wno-format-nonliteral -Werror -g -O0 -c vendor/lib9p/connection.c -o vendor/build/lib9p/connection.o
clang -Weverything -Wno-padded -Wno-gnu-zero-variadic-macro-arguments -Wno-format-nonliteral -Werror -g -O0 -c vendor/lib9p/request.c -o vendor/build/lib9p/request.o
clang -Weverything -Wno-padded -Wno-gnu-zero-variadic-macro-arguments -Wno-format-nonliteral -Werror -g -O0 -c vendor/lib9p/log.c -o vendor/build/lib9p/log.o
clang -Weverything -Wno-padded -Wno-gnu-zero-variadic-macro-arguments -Wno-format-nonliteral -Werror -g -O0 -c vendor/lib9p/hashtable.c -o vendor/build/lib9p/hashtable.o
clang -Weverything -Wno-padded -Wno-gnu-zero-variadic-macro-arguments -Wno-format-nonliteral -Werror -g -O0 -c vendor/lib9p/utils.c -o vendor/build/lib9p/utils.o
clang -Weverything -Wno-padded -Wno-gnu-zero-variadic-macro-arguments -Wno-format-nonliteral -Werror -g -O0 -c vendor/lib9p/sbuf/sbuf.c -o vendor/build/lib9p/sbuf/sbuf.o
clang -Weverything -Wno-padded -Wno-gnu-zero-variadic-macro-arguments -Wno-format-nonliteral -Werror -g -O0 -c vendor/lib9p/transport/socket.c -o vendor/build/lib9p/transport/socket.o
clang -Weverything -Wno-padded -Wno-gnu-zero-variadic-macro-arguments -Wno-format-nonliteral -Werror -g -O0 -c vendor/lib9p/backend/fs.c -o vendor/build/lib9p/backend/fs.o
libtool -static vendor/build/lib9p/pack.o vendor/build/lib9p/connection.o vendor/build/lib9p/request.o vendor/build/lib9p/log.o vendor/build/lib9p/hashtable.o vendor/build/lib9p/utils.o vendor/build/lib9p/sbuf/sbuf.o vendor/build/lib9p/transport/socket.o vendor/build/lib9p/backend/fs.o -o vendor/build/lib9p/lib9p.a
==> Build docker-machine-driver-xhyve...
 CGO_CFLAGS='-I/Users/zchee/go/src/github.com/zchee/docker-machine-driver-xhyve/vendor/lib9p' CGO_LDFLAGS='/Users/zchee/go/src/github.com/zchee/docker-machine-driver-xhyve/vendor/build/lib9p/lib9p.a -L/Users/zchee/go/src/github.com/zchee/docker-machine-driver-xhyve/vendor/lib9p' go build -v -o bin/docker-machine-driver-xhyve -ldflags  -w -s -X github.com/zchee/docker-machine-driver-xhyve/xhyve.GitCommit=0cf6631  github.com/zchee/docker-machine-driver-xhyve
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/log
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/mcnflag
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/docker/pkg/term
github.com/zchee/docker-machine-driver-xhyve/vendor/golang.org/x/crypto/curve25519
github.com/zchee/docker-machine-driver-xhyve/vendor/golang.org/x/crypto/ssh/terminal
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/state
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/version
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/hooklift/xhyve
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/johanneswuerbach/nfsexports
github.com/zchee/docker-machine-driver-xhyve/vmnet
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/mcnutils
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/drivers/plugin/localbinary
github.com/zchee/docker-machine-driver-xhyve/b2d
github.com/zchee/docker-machine-driver-xhyve/vendor/golang.org/x/crypto/ssh
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/ssh
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/drivers
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/drivers/rpc
github.com/zchee/docker-machine-driver-xhyve/xhyve
github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/docker/machine/libmachine/drivers/plugin
github.com/zchee/docker-machine-driver-xhyve
zchee commented 8 years ago

@PerilousApricot If uou want build lib9p.a only, just

make lib9p
> ls -la vendor/build/lib9p                                                                                                                                                                                               (master)
total 232K
drwxr-xr-x 3 zchee staff  102 Feb 26 04:52 backend/
drwxr-xr-x 3 zchee staff  102 Feb 26 04:52 sbuf/
drwxr-xr-x 3 zchee staff  102 Feb 26 04:52 transport/
-rw-r--r-- 1 zchee staff  15K Feb 26 04:52 connection.o
-rw-r--r-- 1 zchee staff 8.0K Feb 26 04:52 hashtable.o
-rw-r--r-- 1 zchee staff 140K Feb 26 04:52 lib9p.a
-rw-r--r-- 1 zchee staff 4.1K Feb 26 04:52 log.o
-rw-r--r-- 1 zchee staff  19K Feb 26 04:52 pack.o
-rw-r--r-- 1 zchee staff  21K Feb 26 04:52 request.o
-rw-r--r-- 1 zchee staff  14K Feb 26 04:52 utils.o
PerilousApricot commented 8 years ago

Hi @zchee - thanks for your patience, I am very unfamiliar with go, so I want to try and do things "the right way" and get the homebrew recipe updated.

I've tried the following two ways, which each fail differently. I can fix it, but I'd rather do it the way that looks better to you.

  1. Build lib9p w/make, then build the driver with "go build":
  patch :DATA                                                                   
  def install                                                                                                                                                        
    (buildpath/"gopath/src/github.com/zchee/docker-machine-driver-xhyve").install Dir["{*,.git,.gitignore}"]
    ENV["GOPATH"] = "#{buildpath}/gopath"                                       
    build_root = buildpath/"gopath/src/github.com/zchee/docker-machine-driver-xhyve"
    cd build_root do                                                            
      if build.head?                                                            
        git_hash = `git rev-parse --short HEAD --quiet`.chomp                   
        git_hash = "HEAD-#{git_hash}"                                           
        system "make", "lib9p"                                                  
      end                                                                       
      system "go", "build", "-x", "-o", bin/"docker-machine-driver-xhyve",      
      "-ldflags",                                                               
      "'-w -s'",                                                                
      "-ldflags",                                                               
      "-X 'github.com/zchee/docker-machine-driver-xhyve/xhyve.GitCommit=Homebrew#{git_hash}'",
      "./main.go"                                                               
    end                                                                         
  end

I had to patch the lib9p.mk file to keep the submodule updates from conflicting with the fact that brew recursively checks out submodules:

diff --git a/mk/lib9p.mk b/mk/lib9p.mk                                          
index e374310..9be0ae7 100644                                                   
--- a/mk/lib9p.mk                                                               
+++ b/mk/lib9p.mk                                                               
@@ -31,7 +31,7 @@ default: build                                                
 lib9p: fetch $(LIB9P_LIB)                                                      

 fetch:                                                                         
-   ${GIT_CMD} submodule update --init                                          
+   [ -d vendor/lib9p ] || ${GIT_CMD} submodule update --init                   
    mkdir -p ${LIB9P_BUILD_DIR} ${LIB9P_BUILD_DIR}/sbuf ${LIB9P_BUILD_DIR}/transport ${LIB9P_BUILD_DIR}/backend

 vendor/build/lib9p/%.o: vendor/lib9p/%.c

This fails with:

vendor/github.com/hooklift/xhyve/pci_virtio_9p.c:49:10: fatal error: 
      'lib9p.h' file not found

Which I can solve with adding the right CGO_FLAGS and CGO_LDFLAGS.

  1. Trying to build with just "make" gives a prompt from the chown, which doesn't mesh with installing via brew properly. I "think" the right thing to do is split up the target like so:
default: fixperms

build: bin/docker-machine-driver-xhyve

bin/docker-machine-driver-xhyve: lib9p
        @test -d bin || mkdir -p bin;
        @echo "${CBLUE}==>${CRESET} Build ${CGREEN}${PACKAGE}${CRESET}..."
        @echo "${CBLACK} CGO_CFLAGS=${CGO_CFLAGS} CGO_LDFLAGS=${CGO_LDFLAGS} ${GO_BUILD} -ldflags "$(GO_LDFLAGS)" ${GO_GCFLAGS} ${TOP_PACKAGE_DIR}/${PACKAGE} ${CRESET}"; \
        CGO_CFLAGS=${CGO_CFLAGS} CGO_LDFLAGS=${CGO_LDFLAGS} ${GO_BUILD} -ldflags "$(GO_LDFLAGS)" ${GO_GCFLAGS} ${TOP_PACKAGE_DIR}/${PACKAGE} || exit 1

fixperms: bin/docker-machine-driver-xhyve
        @echo "${CBLUE}==>${CRESET} Change ${CGREEN}${PACKAGE}${CRESET} binary owner and group to root:wheel. Please root password${CRESET}"; \
        sudo chown root:wheel ${OUTPUT} && sudo chmod u+s ${OUTPUT}

install: fixperms
        @echo "${CBLUE}==>${CRESET} Install ${CGREEN}${PACKAGE}${CRESET}..."
        @test -d /usr/local/bin || mkdir -p /usr/local/bin
        sudo cp -p ./bin/docker-machine-driver-xhyve /usr/local/bin/

This way, the makefile doesn't call sudo by default.

Sorry for it being long, I want to make sure what I'm doing jives with what you want.

zchee commented 8 years ago

@PerilousApricot Thanks reply!

I will consider, and comment later.

PerilousApricot commented 8 years ago

No rush! Just trying to contribute w/what seems like an incredibly useful project :)