Documentation on cross-compilation

[goloroden]

Today, I'd like to ask for a small feature request, which – I think – could be useful to more people than just me.

It's about cross-compiling Go programs which shall use DuckDB. While building a binary for the platform you're developing on is pretty simple, doing the same with cross-compilation is not.

Given that there are three major operating systems these days (macOS, Linux, Windows), and two CPU architectures (x86-64 and ARM), you end up with quite a number of combinations: You essentially have six combinations as development base, and you have six combinations you might want to build.

E.g.:

• Build for macOS / ARM on a Linux / x86-64 machine.
• Build for Windows / x86-64 on a macOS / arm machine.
• And so on …

Effectively, this is 36 combinations in total.

Now here's my question: Since I find it pretty difficult to figure out how to cross-compile, would it maybe be possible to add information on this to the documentation of this module? I think that it doesn't need an example per combination, but having some general guidelines or tips would already be helpful (or even a link to a page which explains it in more detail).

What do you think of this?

PS: If someone is able to provide the commands, I'd be more than happy to work on the documentation and submit a PR. It's just that I don't know how to do it myself.

[marcboeker]

@goloroden Thanks for bringing this up. Cross-compiling is a total pain. What could make sense is an issue where we first collect all the cross-compiling combinations we have working so far. Then we can start generating a kind of matrix from it with source and destination and which toolchain to use.

Starting with macOS we could use the approach I've commented in an earlier issue you have opened.

With linux as source we could use the compile chain from the Github runners (need to figure out what the have pre-installed) and the commands from the Makefile.

From and to Windows: @taniabogatsch is currently working on a Windows build. But I have no access to Windows to test anything. And I'm still confused about Mingw32, Msys2, Cygwin...

[goloroden]

Okay, I'll start collecting things here. This is my test program:

package main

import (
    "database/sql"
    "fmt"

    _ "github.com/marcboeker/go-duckdb"
)

func main() {
    db, err := sql.Open("duckdb", "")
    if err != nil {
        panic(err)
    }
    defer db.Close()

    var value int
    row := db.QueryRow("SELECT 23 AS value")
    row.Scan(&value)
    fmt.Println(value)
}

The goal is to compile and run the binary afterwards in the appropriate environment. If the file size is around 40 MByte, it apparently includes DuckDB statically linked; and if it runs and prints 23, it seems to work.

This list can – sooner or later – maybe be merged into the official documentation, to make things easier for other people who want to do cross-compilation with DuckDB and Go. If you want to add an entry to this list, please leave a comment below, outlining on which OS/Arch you are compiling, for which OS/Arch you are compiling, what the required steps are, and whether it was verified to work.

So far, the following combinations have been reported and successfully verified to work:

✅ On Linux (ARM) for Linux (ARM)

(Verified by @goloroden)

# Compile
$ go build .

❌ On Linux (ARM) for Linux (x86)

❌ On Linux (ARM) for macOS (ARM)

❌ On Linux (ARM) for macOS (x86)

❌ On Linux (x86) for Linux (ARM)

✅ On Linux (x86) for Linux (x86)

(Verified by @goloroden)

# Compile
$ go build .

❌ On Linux (x86) for macOS (ARM)

❌ On Linux (x86) for macOS (x86)

✅ On macOS (ARM) for Linux (ARM)

(Verified by @goloroden)

# Install the cross-compiler components for Linux (ARM)
$ brew install messense/macos-cross-toolchains/aarch64-unknown-linux-gnu

# Cross-compile
$ CC="aarch64-linux-gnu-gcc" CXX="aarch64-linux-gnu-g++" CGO_ENABLED=1 GOOS=linux GOARCH=arm64 go build .

✅ On macOS (ARM) for Linux (x86)

(Verified by @goloroden)

# Install the cross-compiler components for Linux (x86)
$ brew install messense/macos-cross-toolchains/x86_64-unknown-linux-gnu

# Cross-compile
$ CC="x86_64-linux-gnu-gcc" CXX="x86_64-linux-gnu-g++" CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build .

✅ On macOS (ARM) for macOS (ARM)

(Verified by @goloroden)

# Compile
$ go build .

✅ On macOS (ARM) for macOS (x86)

(Verified by @goloroden)

# Cross-compile
$ CGO_ENABLED=1 GOOS=darwin GOARCH=amd64 go build .

❌ On macOS (x86) for Linux (ARM)

❌ On macOS (x86) for Linux (x86)

❌ On macOS (x86) for macOS (ARM)

❌ On macOS (x86) for macOS (x86)

[goloroden]

@marcboeker I have collected what I have so far, do you maybe have other ones you can contribute?