Dockerfile: xgo compile for darwin architecture fails

[ncw]

When I try an xgo compile

xgo -image=billziss/xgo-cgofuse -targets=darwin/amd64 github.com/billziss-gh/cgofuse/fuse

I get this error

Checking docker installation...
Client:
 Version:           19.03.8
 API version:       1.40
 Go version:        go1.13.8
 Git commit:        afacb8b7f0
 Built:             Wed Mar 11 23:42:35 2020
 OS/Arch:           linux/amd64
 Experimental:      true

Server:
 Engine:
  Version:          19.03.8
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.8
  Git commit:       afacb8b7f0
  Built:            Wed Mar 11 22:48:33 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.3.3-0ubuntu2
  GitCommit:        
 runc:
  Version:          spec: 1.0.1-dev
  GitCommit:        
 docker-init:
  Version:          0.18.0
  GitCommit:        

Checking for required docker image billziss/xgo-cgofuse... found.
Cross compiling github.com/billziss-gh/cgofuse/fuse...
Fetching main repository github.com/billziss-gh/cgofuse/fuse...
github.com/billziss-gh/cgofuse (download)
Compiling for darwin-10.6/amd64...
# github.com/billziss-gh/cgofuse/fuse
./host_cgo.go:390:6: error: no member named 'nonseekable' in 'struct fuse_file_info'
        fi->nonseekable = nonseekable;
        ~~  ^
1 error generated.
2020/06/29 11:55:41 Failed to cross compile package: exit status 2.

I bisected the problem and (unsuprisingly!) this was the commit that broke it 8e41a0ce969b7c07d55751a9ad4e1941f285f620 - this was new to v1.3.0

So I think the FUSE headers need to be updated somehow in the Docker file for darwin.

https://github.com/billziss-gh/cgofuse/blob/f414e61c21ae71e205e28156b55093867fb9b458/Dockerfile#L14-L24

I note that 2.8.3 is a way behind the current 3.10.6

So I think either we need to do one of these

• update osxfuse
• #ifdef out that feature on osx

I'm not sure which of those is the best way forward though!

[ncw]

I've decided to drop this build from rclone for the moment as it is only used for testing, so no urgency!

[billziss-gh]

Hey, Nick. Sorry for the delay in responding. I will have a look at this tomorrow.

[billziss-gh]

I believe the right thing here is to update the headers used in the Docker image.

I note that OSXFUSE 3.x series still uses the FUSE2 API, so it should be ok to update to latest OSXFUSE.

[billziss-gh]

So the docker image has now been modified to include OSXFUSE 3.8.3 which includes the nonseekable symbol. (I chose OSXFUSE 3.8.3 and not the latest version because this is the last version that 7z can handle; newer versions of OSXFUSE use signed DMG's which 7z cannot handle.)

Unfortunately this is now failing during apt-get install because of missing files. The error message suggests that I should run apt-get update, which I already am except in an earlier Dockerfile step. I suspect that this is an effect of the Docker cache:

• This RUN step is pulled from cache:

  Step 3/7 : RUN dpkg --add-architecture i386 && dpkg --add-architecture amd64 && apt-get update && apt-get install -y --no-install-recommends p7zip-full
  ---> Using cache
  ---> c1143cf251eb

• This RUN step is being run because it came after the OSXFUSE RUN step which was just modified. Unfortunately it seems to fail because it needs a fresh apt-get update.

  Step 5/7 : RUN apt-get install -y --no-install-recommends libfuse-dev:i386 && apt-get install -y --no-install-recommends libfuse-dev:amd64 && apt-get download libfuse-dev:i386 && dpkg -x libfuse-dev*i386*.deb /
  ---> Running in 407faefdbff8
  ...
  Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

Is there a way to trigger DockerHub builds without the build cache? I no longer have easy access to Docker for Linux containers and I may not be able to fix this timely.

[billziss-gh]

Ok, so I tried the kludge of adding a second apt-get update in the LIBFUSE installation RUN step and the build has now succeeded.

Can you please verify that the resulting xgo image works for you? Try pulling the latest tag for now.

[ncw]

I can confirm that that fixes the rclone builds - thanks Bill :-)

Sorry you had to debug on DockerHub - that is a miserable experience.

[billziss-gh]

Excellent. I am closing this and will follow up on any additional work via rclone/rclone#4393.