Connect to docker hub to auto build the container

[ncw]

Is there any chance you could connect this repo to the docker hub to make automated builds of the container?

I did it recently with rclone and the process was quite straight forward :-)

It makes a build on every push to the repo and you can set it to build when upstream containers change too.

Thanks

Nick

[billziss-gh]

@ncw this looks interesting, but I currently have too much work on other projects to look into this more closely. If you need this in a hurry and you already have the know-how would you consider submitting a PR? (I know you are probably too busy with rclone, etc. yourself.)

[ncw]

It isn't something you can do with a PR it needs permissions granting on this repo to the docker hub organization. After that it is just clicking some buttons!

I could add it to the rclone docker hub but I think you'd need to make me a member of this repo.

My plan B was to fork this repo (under github.com/rclone) and set up an auto build on the rclone docker hub which will suit my purposes for building rclone and others could find the docker image there if they wanted. I'll do this anyway and post a link so you can see what it looks like.

[billziss-gh]

@ncw, I will look into this today.

[billziss-gh]

BTW, dumb question, but by "container" you mean "xgo-cgofuse". Is this correct?

EDIT: DockerHub seems painfully slow for me. I have just started the build and after 10 minutes it is still queued.

[billziss-gh]

This eventually has finished successfully:

https://hub.docker.com/r/billziss/xgo-cgofuse

I probably should add some tests to verify that the resulting docker file works indeed.

Let me know if this is what you wanted. I will devote a few more hours on cgofuse to clean it up and make sure that it properly builds on the different CI systems.

[billziss-gh]

I believe this is now complete and new xgo-cgofuse images are built after every commit to master.

While I was at it I ended up cleaning up the project and its README and made sure that it works on all CI systems (most CI build were broken because of changes in the CI environment -- e.g. new OS versions that restrict loading kexts on macOS, etc.)

Let me know if this is what you wanted, so we can close this.

[ncw]

That is perfect, thanks Bill :-)

Yes docker hub seems very slow! However I was told by an rclone contributor that builds done by docker hub are preferred by many people because of the assurances they give.

I'll plug your new container into the rclone build process.

Did you tick the rebuild if the upstream changes button? That would need useful here I think.

You could put a button like this in the readme

[billziss-gh]

Did you tick the rebuild if the upstream changes button? That would need useful here I think.

Where is this setting? I am fairly certain I saw it yesterday, but can no longer find it today.

EDIT: Ok, found it:

BTW, there is an open issue that suggests this may not work as intended. See

https://github.com/docker/hub-feedback/issues/ 1717

[ncw]

BTW, there is an open issue that suggests this may not work as intended. See

:-(

Well hopefully it will start working soon!

[billziss-gh]

Well hopefully it will start working soon!

We can try an experiment if you want.

---

BTW, while I was working on cgofuse I considered the idea of making !cgo builds for OS'es other than Windows. After some research I came up with this:

package main

import (
    "unsafe"
)

//go:cgo_import_dynamic libc_puts_ puts "/usr/lib/libSystem.B.dylib"
//go:cgo_import_dynamic _ _ "/usr/lib/libSystem.B.dylib"

//go:linkname libc_puts_ libc_puts_
var libc_puts_ byte
var libc_puts = &libc_puts_

//go:linkname syscall_syscall syscall.syscall
func syscall_syscall(fn, a1, a2, a3 uintptr) (r1, r2, err uintptr)

func main() {
    p := uintptr(unsafe.Pointer(libc_puts))

    var s [7]byte
    s[0] = 'h'
    s[1] = 'e'
    s[2] = 'l'
    s[3] = 'l'
    s[4] = 'o'
    s[5] = '\n'
    s[6] = 0
    t := uintptr(unsafe.Pointer(&s))

    syscall_syscall(p, t, 0, 0)
}

This builds and prints hello on my Mac. It requires Go 1.12 (I think one of the later Go 1.11 versions works as well.

It could become the basis of a !cgo variant for macOS. I would also need an equivalent of the Windows syscall.NewCallbackCDecl.

[ncw]

That is an interesting idea.. Those go: pragmas aren't very well documented are they?

So does it make a static exe at the end?

[billziss-gh]

That is an interesting idea.. Those go: pragmas aren't very well documented are they?

Some of them are (e.g. go:linkname), but one mostly figures them out by reading the Go sources.

So does it make a static exe at the end?

I do not believe you can make a static exec and use dlopen at the same time. But at least on macOS, go executables always dynamically link to the system libs so that is not a problem on that OS.

I also do not know how to make this work on Linux; I could not find an equivalent syscall.syscall internal function for it. The exported syscall.Syscall does a real system call and cannot be used to call arbitrary C functions.

All this machinery is already built into the go runtime, but it is only used by cgo (e.g. asmcgocall). I believe that the go team should refactor these parts and allow them to be used by non-cgo programs.

[ncw]

Do you think this would give a significant advantage over not using cgo? I guess it still has the same syscall overhead and need for dynamic libraries, however it doesn't need a C compiler - would you agree with that?

I haven't tried out the Windows nocgo interface yet - do you think that would be a good choice for rclone?

BTW I plugged the new container into rclone's build process - it is working very well - thanks!

[billziss-gh]

I think not needing a C compiler would be the big win here.

I haven't tried out the Windows nocgo interface yet - do you think that would be a good choice for rclone?

It depends on how you use cgofuse. If you use it only for Windows, then it might allow you to stop using cgo and go back to go (i.e. nocgo) builds. If you use it for other platforms, then IMO it is not worth changing, because you need a C compiler on those platforms anyway.

[ncw]

It depends on how you use cgofuse. If you use it only for Windows, then it might allow you to stop using cgo and go back to go (i.e. nocgo) builds. If you use it for other platforms, then IMO it is not worth changing, because you need a C compiler on those platforms anyway.

I build the Windows binary natively on Windows so it probably isn't worth switching for rclone. It is a very welcome development though and I'll use it for my next go/fuse project for definite!