search

bazelbuild / bazel-central-registry

The central registry of Bazel modules for the Bzlmod external dependency system.
https://registry.bazel.build
Apache License 2.0
263 stars 342 forks source link

wanted: marcboeker/go-duckdb #2517

Open derekperkins opened 4 months ago

derekperkins commented 4 months ago

Module location

https://github.com/marcboeker/go-duckdb

Link to bzlmod issue in the module's repository

No response

Any other context to provide?

We just worked with @pcj to get go-duckdb building, and I wanted to make it publicly available in case anyone else is interested. This appears to be similar to https://github.com/bazelbuild/bazel-central-registry/tree/main/modules/circl

com_github_apache_arrow_go_v14_arrow_cdata.patch com_github_marcboeker_go_duckdb.libduckdb.patch README.md

Fund our work

derekperkins commented 4 months ago

patches

This directory contains patch files for selected bazel external workspaces.

patches/com_github_apache_arrow_go_v14_arrow_cdata.patch

This patch is needed because gazelle / go_repository does not generate correct targets for cgo-related header files that are in subdirectories of a build file.

patches/com_github_marcboeker_go_duckdb.libduckdb.patch

This patch is needed because gazelle looks for lines like #cgo darwin,arm64 LDFLAGS: -lc++ -L${SRCDIR}/deps/darwin_arm64 in source files, but it transforms the ${SRCDIR} improperly in the context of go_repository wherein it should be something like external/{REPOSITORY_NAME}/${SRCDIR}.

Also, gazelle does not automatically generate cc_library rules for precompiled libduckdb.a files.

Regenerating Patches

To recreate the patch file, use your preferred technique to do so. One way is as follows:

  1. Ensure the external workspace exists on the filesystem: bazel query @com_github_apache_arrow_go_v14//...
  2. Open a code editor in the external workspace: (cd $(bazel info output_base)/external/com_github_apache_arrow_go_v14 && code .)
  3. Open an integrated terminal in the root directory of that external workspace.
  4. Initialize a git repo with the minimal set of files you are interested in: git init && git add arrow/cdata && git commit -m 'initial state'
  5. Edit the BUILD.bazel file (or files). You may need to run bazel sync --configure or similar command to see build file changes in the default workspace.
  6. Commit the changes and copy the : git add -u && git commit -m 'updated' && git show {COMMIT} | pbcopy, or just copy the diff to clipboard: git diff | pbcopy.
  7. Paste the contents into the .patch file.
  8. Ensure the patch file is named in an exports_files rule (needed for repository rule).
  9. Make sure the go_repository names the patch and patch_args attributes. patch_args = ["p1"] means "strip the first path segment off the filename" (e.g. a/arrow/cdata/BUILD.bazel does not exist, but arrow/cdata/BUILD.bazel does).

com_github_apache_arrow_go_v14_arrow_cdata.patch

--- a/arrow/cdata/BUILD.bazel
+++ b/arrow/cdata/BUILD.bazel
@@ -1,4 +1,4 @@
-load("@io_bazel_rules_go//go:def.bzl", "go_library")
+load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_source")

 go_library(
     name = "cdata",
@@ -8,6 +8,7 @@ go_library(
         "exports.go",
         "interface.go",
         "trampoline.c",
+        ":headers",
     ],
     cgo = True,
     importpath = "github.com/apache/arrow/go/v14/arrow/cdata",
@@ -31,3 +32,11 @@ alias(
     actual = ":cdata",
     visibility = ["//visibility:public"],
 )
+
+go_source(
+    name = "headers",
+    srcs = [
+        "arrow/c/abi.h",
+        "arrow/c/helpers.h",
+    ],
+)

com_github_marcboeker_go_duckdb.libduckdb.patch

diff --git a/BUILD.bazel b/BUILD.bazel
index 8285865..37e192e 100644
--- a/BUILD.bazel
+++ b/BUILD.bazel
@@ -18,49 +18,83 @@ go_library(
         "transaction.go",
         "types.go",
     ],
+    cdeps = select({
+        "@io_bazel_rules_go//go/platform:android_amd64": [
+            "//deps/linux_amd64:libduckdb",
+        ],
+        "@io_bazel_rules_go//go/platform:android_arm64": [
+            "//deps/linux_arm64:libduckdb",
+        ],
+        "@io_bazel_rules_go//go/platform:darwin_386": [
+        ],
+        "@io_bazel_rules_go//go/platform:darwin_amd64": [
+            "//deps/darwin_amd64:libduckdb",
+        ],
+        "@io_bazel_rules_go//go/platform:darwin_arm": [
+        ],
+        "@io_bazel_rules_go//go/platform:darwin_arm64": [
+            "//deps/darwin_arm64:libduckdb",
+        ],
+        "@io_bazel_rules_go//go/platform:freebsd_amd64": [
+            "//deps/freebsd_amd64:libduckdb",
+        ],
+        "@io_bazel_rules_go//go/platform:ios_amd64": [
+            "//deps/darwin_amd64:libduckdb",
+        ],
+        "@io_bazel_rules_go//go/platform:ios_arm64": [
+            "//deps/darwin_arm64:libduckdb",
+        ],
+        "@io_bazel_rules_go//go/platform:linux_amd64": [
+            "//deps/linux_amd64:libduckdb",
+        ],
+        "@io_bazel_rules_go//go/platform:linux_arm64": [
+            "//deps/linux_arm64:libduckdb",
+        ],
+        "//conditions:default": [],
+    }),
     cgo = True,
     clinkopts = select({
         "@io_bazel_rules_go//go/platform:android_amd64": [
             "-lduckdb",
-            "-lstdc++ -lm -ldl -Ldeps/linux_amd64",
+            "-lstdc++ -lm -ldl -Lexternal/com_github_marcboeker_go_duckdb/deps/linux_amd64",
         ],
         "@io_bazel_rules_go//go/platform:android_arm64": [
             "-lduckdb",
-            "-lstdc++ -lm -ldl -Ldeps/linux_arm64",
+            "-lstdc++ -lm -ldl -Lexternal/com_github_marcboeker_go_duckdb/deps/linux_arm64",
         ],
         "@io_bazel_rules_go//go/platform:darwin_386": [
             "-lduckdb",
         ],
         "@io_bazel_rules_go//go/platform:darwin_amd64": [
-            "-lc++ -Ldeps/darwin_amd64",
+            "-lc++ -Lexternal/com_github_marcboeker_go_duckdb/deps/darwin_amd64",
             "-lduckdb",
         ],
         "@io_bazel_rules_go//go/platform:darwin_arm": [
             "-lduckdb",
         ],
         "@io_bazel_rules_go//go/platform:darwin_arm64": [
-            "-lc++ -Ldeps/darwin_arm64",
+            "-lc++ -Lexternal/com_github_marcboeker_go_duckdb/deps/darwin_arm64",
             "-lduckdb",
         ],
         "@io_bazel_rules_go//go/platform:freebsd_amd64": [
             "-lduckdb",
-            "-lstdc++ -lm -ldl -Ldeps/freebsd_amd64",
+            "-lstdc++ -lm -ldl -Lexternal/com_github_marcboeker_go_duckdb/deps/freebsd_amd64",
         ],
         "@io_bazel_rules_go//go/platform:ios_amd64": [
             "-lc++ -Ldeps/darwin_amd64",
             "-lduckdb",
         ],
         "@io_bazel_rules_go//go/platform:ios_arm64": [
-            "-lc++ -Ldeps/darwin_arm64",
+            "-lc++ -Lexternal/com_github_marcboeker_go_duckdb/deps/darwin_arm64",
             "-lduckdb",
         ],
         "@io_bazel_rules_go//go/platform:linux_amd64": [
             "-lduckdb",
-            "-lstdc++ -lm -ldl -Ldeps/linux_amd64",
+            "-lstdc++ -lm -ldl -Lexternal/com_github_marcboeker_go_duckdb/deps/linux_amd64",
         ],
         "@io_bazel_rules_go//go/platform:linux_arm64": [
             "-lduckdb",
-            "-lstdc++ -lm -ldl -Ldeps/linux_arm64",
+            "-lstdc++ -lm -ldl -Lexternal/com_github_marcboeker_go_duckdb/deps/linux_arm64",
         ],
         "//conditions:default": [],
     }),
diff --git a/deps/darwin_amd64/BUILD.bazel b/deps/darwin_amd64/BUILD.bazel
index 471671e..4f508b6 100644
--- a/deps/darwin_amd64/BUILD.bazel
+++ b/deps/darwin_amd64/BUILD.bazel
@@ -12,3 +12,9 @@ alias(
     actual = ":darwin_amd64",
     visibility = ["//visibility:public"],
 )
+
+cc_library(
+    name = "libduckdb",
+    srcs = ["libduckdb.a"],
+    visibility = ["//visibility:public"],
+)
diff --git a/deps/darwin_arm64/BUILD.bazel b/deps/darwin_arm64/BUILD.bazel
index edc81b3..848fe8b 100644
--- a/deps/darwin_arm64/BUILD.bazel
+++ b/deps/darwin_arm64/BUILD.bazel
@@ -12,3 +12,9 @@ alias(
     actual = ":darwin_arm64",
     visibility = ["//visibility:public"],
 )
+
+cc_library(
+    name = "libduckdb",
+    srcs = ["libduckdb.a"],
+    visibility = ["//visibility:public"],
+)
diff --git a/deps/freebsd_amd64/BUILD.bazel b/deps/freebsd_amd64/BUILD.bazel
index 849a7e3..f347669 100644
--- a/deps/freebsd_amd64/BUILD.bazel
+++ b/deps/freebsd_amd64/BUILD.bazel
@@ -12,3 +12,9 @@ alias(
     actual = ":freebsd_amd64",
     visibility = ["//visibility:public"],
 )
+
+cc_library(
+    name = "libduckdb",
+    srcs = ["libduckdb.a"],
+    visibility = ["//visibility:public"],
+)
diff --git a/deps/linux_amd64/BUILD.bazel b/deps/linux_amd64/BUILD.bazel
index 0acbb38..23be608 100644
--- a/deps/linux_amd64/BUILD.bazel
+++ b/deps/linux_amd64/BUILD.bazel
@@ -12,3 +12,9 @@ alias(
     actual = ":linux_amd64",
     visibility = ["//visibility:public"],
 )
+
+cc_library(
+    name = "libduckdb",
+    srcs = ["libduckdb.a"],
+    visibility = ["//visibility:public"],
+)
diff --git a/deps/linux_arm64/BUILD.bazel b/deps/linux_arm64/BUILD.bazel
index 64200d5..5c7fba0 100644
--- a/deps/linux_arm64/BUILD.bazel
+++ b/deps/linux_arm64/BUILD.bazel
@@ -12,3 +12,9 @@ alias(
     actual = ":linux_arm64",
     visibility = ["//visibility:public"],
 )
+
+cc_library(
+    name = "libduckdb",
+    srcs = ["libduckdb.a"],
+    visibility = ["//visibility:public"],
+)
fmeum commented 4 months ago

Thanks for working on this! Feel free to submit a patched module just like circl, I can review it.

That said, I am planning to adapt the clinkopts logic in Gazelle to make one half of the patch unnecessary in future releases.

derekperkins commented 4 months ago

Submitted. We don't use bzlmod yet, and I don't have much bazel experience. I just wanted to share this for anyone else interested.