Closed amos-lsl closed 4 months ago
marcboeker
commented
1 year ago Windows seems to be problematic. I've hadn't had success yet. Maybe you could try dynamic linking instead of compiling the amalgamation file. You could try something like:
Download the latest release from DuckDB for Windows and run the following:
CGO_ENABLED=1 CGO_LDFLAGS="-L/path/to/duckdb.dll" go build -tags=duckdb_use_lib main.go
LD_LIBRARY_PATH=/path/to/libs ./main
amos-lsl
commented
1 year ago Windows seems to be problematic. I've hadn't had success yet. Maybe you could try dynamic linking instead of compiling the amalgamation file. You could try something like:
Download the latest release from DuckDB for Windows and run the following:
CGO_ENABLED=1 CGO_LDFLAGS="-L/path/to/duckdb.dll" go build -tags=duckdb_use_lib main.go LD_LIBRARY_PATH=/path/to/libs ./main
i have try this, but run error, like others who tried in windows
Dimirolz
commented
1 year ago Windows seems to be problematic. I've hadn't had success yet. Maybe you could try dynamic linking instead of compiling the amalgamation file. You could try something like:
Download the latest release from DuckDB for Windows and run the following:
CGO_ENABLED=1 CGO_LDFLAGS="-L/path/to/duckdb.dll" go build -tags=duckdb_use_lib main.go LD_LIBRARY_PATH=/path/to/libs ./main
Dynamic linking is working for my, btw.
marcboeker
commented
1 year ago I've done extensive work on getting it to link statically on Windows, but had no success. The main problem is, that CGO complains about missing __imp_ prefixed symbols.
undefined reference to `__imp_.....`They are available in the dll but not in the lib which is btw. implausibly small (~200kb).
ZenLiuCN
commented
10 months ago This seems a solution, I mentioned under #51 . Current a success workflow is:
deps/windows_amd64
vendor.go
// Package windows_amd64 is required to provide support for vendoring modules
// DO NOT REMOVE
package windows_amd64create a file under source root cgo_static_win.go, cause it have many different that linux or darwin
//go:build !duckdb_use_lib && windows && amd64
package duckdb
/*
#cgo LDFLAGS: -lduckdb_static -lws2_32 -lduckdb_fsst -lduckdb_fmt -lduckdb_pg_query -lduckdb_re2 -lduckdb_miniz -lduckdb_utf8proc -lduckdb_hyperloglog -lduckdb_fastpforlib -lduckdb_mbedtls -lparquet_extension -lduckdb_static -Wl,-Bstatic -lstdc++ -Wl,-Bstatic -lpthread -lm -L${SRCDIR}/deps/windows_amd64
#include <duckdb.h>
*/
import "C"Most important changs on duckdb.h at line 18
//===----------------------------------------------------------------------===//
//
// DuckDB
//
// duckdb.h
//
//
//===----------------------------------------------------------------------===//
#pragma once
// duplicate of duckdb/main/winapi.hpp
#ifndef DUCKDB_API
#ifdef _WIN32
#if defined(DUCKDB_BUILD_LIBRARY) && !defined(DUCKDB_BUILD_LOADABLE_EXTENSION)
#define DUCKDB_API __declspec(dllexport)
#else
// line 18 is patched
#define DUCKDB_API //__declspec(dllimport) !! comment out this part
#endif
#else
#define DUCKDB_API
#endif
#endifNotes:
ws2_32 is required by libparquet_extension-lparquet_extension -lduckdb_static cause that parquet_extension also requires duckdb_static-Wl,-Bstatic -lstdc++ -Wl,-Bstatic -lpthread to avoid ship with extra dll (libstdc++-6.dll, libwinpthread-1.dll, libgcc_s_seh-1.dll)Also I tested the example (within the engine ), seems that all functional.
2023/12/17 20:09:17 INFO [{"current_setting('access_mode')":"read_write"}]
2023/12/17 20:09:17 INFO {"lastInsertId":0,"rowsAffected":0}
2023/12/17 20:09:17 INFO {"lastInsertId":0,"rowsAffected":1}
2023/12/17 20:09:17 INFO {"rowsAffected":1,"lastInsertId":0}
2023/12/17 20:09:17 INFO [{"age":99,"height":1.909999966621399,"awesome":true,"bday":"1970-01-01T00:00:00Z","name":"marc"},{"height":1.850000023841858,"awesome":true,"bday":"1951-01-23T00:00:00Z","name":"macgyver","age":70}]Test Results:
marcboeker
commented
10 months ago @ZenLiuCN Thanks for the instructions! Short update: I'm working on integrating your suggestions into the build process. See #140 There are still some problems, I need to figure out.
marcboeker
commented
10 months ago @ZenLiuCN I've tried to integrate the go-duckdb windows deps from your engine repo but I'm getting
go test -v -race -count=1 .
# github.com/marcboeker/go-duckdb.test
C:\Program Files\Go\pkg\tool\windows_amd64\link.exe: running gcc failed: exit status 1
C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/Marc/Desktop/go-duckdb/deps/windows_amd64/libduckdb_static.a(logical_types-c.cpp.obj):logical_types-c.cpp:(.text+0xbfe): undefined reference to `std::__throw_bad_array_new_length()'
C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/Marc/Desktop/go-duckdb/deps/windows_amd64/libduckdb_static.a(logical_types-c.cpp.obj):logical_types-c.cpp:(.text+0x1187): undefined reference to `std::__throw_bad_array_new_length()'
C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/Marc/Desktop/go-duckdb/deps/windows_amd64/libduckdb_static.a(prepared-c.cpp.obj):prepared-c.cpp:(.text$_ZNSt10_HashtableINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4pairIKS5_N6duckdb5ValueEESaISA_ENSt8__detail10_Select1stENS8_29CaseInsensitiveStringEqualityENS8_33CaseInsensitiveStringHashFunctionENSC_18_Mod_range_hashingENSC_20_Default_ranged_hashENSC_20_Prime_rehash_policyENSC_17_Hashtable_traitsILb1ELb0ELb1EEEE9_M_rehashEyRKy[_ZNSt10_HashtableINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4pairIKS5_N6duckdb5ValueEESaISA_ENSt8__detail10_Select1stENS8_29CaseInsensitiveStringEqualityENS8_33CaseInsensitiveStringHashFunctionENSC_18_Mod_range_hashingENSC_20_Default_ranged_hashENSC_20_Prime_rehash_policyENSC_17_Hashtable_traitsILb1ELb0ELb1EEEE9_M_rehashEyRKy]+0x108): undefined reference to `std::__throw_bad_array_new_length()'
C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/_/Desktop/go-duckdb/deps/windows_amd64/libduckdb_static.a(result-c.cpp.obj):result-c.cpp:(.text+0xec0): undefined reference to `std::__throw_bad_array_new_length()'
C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/_/Desktop/go-duckdb/deps/windows_amd64/libduckdb_static.a(result-c.cpp.obj):result-c.cpp:(.text+0xf01): undefined reference to `std::__throw_bad_array_new_length()'
C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/_/Desktop/go-duckdb/deps/windows_amd64/libduckdb_static.a(result-c.cpp.obj):result-c.cpp:(.text+0x1460): more undefined references to `std::__throw_bad_array_new_length()' follow
collect2.exe: error: ld returned 1 exit statusdeps/windows_amd64 contains the following:
Directory: C:\Users\Dev\Desktop\go-duckdb\deps\windows_amd64
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 22/12/2023 16:29 490 build.sh
-a--- 22/12/2023 16:29 62 copyLib.sh
-a--- 22/12/2023 16:29 171252 libautocomplete_extension.a
-a--- 22/12/2023 16:29 219416 libduckdb_fastpforlib.a
-a--- 22/12/2023 16:29 655098 libduckdb_fmt.a
-a--- 22/12/2023 16:29 45386 libduckdb_fsst.a
-a--- 22/12/2023 16:29 40218 libduckdb_hyperloglog.a
-a--- 22/12/2023 16:29 136960 libduckdb_mbedtls.a
-a--- 22/12/2023 16:29 105992 libduckdb_miniz.a
-a--- 22/12/2023 16:29 494436 libduckdb_pg_query.a
-a--- 22/12/2023 16:29 821462 libduckdb_re2.a
-a--- 22/12/2023 16:29 81849598 libduckdb_static.a
-a--- 22/12/2023 16:29 311942 libduckdb_utf8proc.a
-a--- 22/12/2023 16:29 459490 libexcel_extension.a
-a--- 22/12/2023 16:29 1047314 libfts_extension.a
-a--- 22/12/2023 16:29 19421390 libicu_extension.a
-a--- 22/12/2023 16:29 97340 libinet_extension.a
-a--- 22/12/2023 16:29 2705052 libjson_extension.a
-a--- 22/12/2023 16:29 4036218 libparquet_extension.a
-a--- 22/12/2023 16:29 1759702 libsqlsmith_extension.a
-a--- 22/12/2023 16:29 2593182 libtpcds_extension.a
-a--- 22/12/2023 16:29 1186888 libtpch_extension.a
-a--- 22/12/2023 16:29 199 mergeStaticLib.sh
-a--- 22/12/2023 16:29 872 vendor.goAnd the duckdb.h is also modified by removing __declspec(dllimport).
Do you have any idea why there are still missing references? Do the tests work for you? Even running make test inside engine/sqlx/duckdb/go-duckdb yields the same errors as above.
ZenLiuCN
commented
10 months ago I was use mingw64_posix_seh. Does it the same? PS use msvc maybe a better option, I can't get mysql fdw to compiled by mingw.
ZenLiuCN
commented
10 months ago @marcboeker 13.2.0 is the gcc version which I used. I saw that you was using 10.3.0. It may the difference. Notes:
go test -v -race -count=1 .
=== RUN TestAppender
--- PASS: TestAppender (0.68s)
=== RUN TestOpen
=== PAUSE TestOpen
=== RUN TestConnPool
--- PASS: TestConnPool (0.01s)
=== RUN TestConnInit
--- PASS: TestConnInit (7.10s)
=== RUN TestExec
=== PAUSE TestExec
=== RUN TestQuery
=== PAUSE TestQuery
=== RUN TestStruct
=== PAUSE TestStruct
=== RUN TestMap
=== PAUSE TestMap
=== RUN TestList
=== PAUSE TestList
=== RUN TestDecimal
=== PAUSE TestDecimal
=== RUN TestUUID
=== PAUSE TestUUID
=== RUN TestENUMs
=== PAUSE TestENUMs
=== RUN TestHugeInt
=== PAUSE TestHugeInt
=== RUN TestVarchar
=== PAUSE TestVarchar
=== RUN TestBlob
=== PAUSE TestBlob
=== RUN TestBoolean
=== PAUSE TestBoolean
=== RUN TestJSON
=== PAUSE TestJSON
=== RUN TestDate
=== PAUSE TestDate
=== RUN TestTimestamp
=== PAUSE TestTimestamp
=== RUN TestInterval
=== PAUSE TestInterval
=== RUN TestEmpty
=== PAUSE TestEmpty
=== RUN TestTypeNamesAndScanTypes
=== RUN TestTypeNamesAndScanTypes/BOOLEAN
=== RUN TestTypeNamesAndScanTypes/TINYINT
=== RUN TestTypeNamesAndScanTypes/SMALLINT
=== RUN TestTypeNamesAndScanTypes/INTEGER
=== RUN TestTypeNamesAndScanTypes/BIGINT
=== RUN TestTypeNamesAndScanTypes/UTINYINT
=== RUN TestTypeNamesAndScanTypes/USMALLINT
=== RUN TestTypeNamesAndScanTypes/UINTEGER
=== RUN TestTypeNamesAndScanTypes/UBIGINT
=== RUN TestTypeNamesAndScanTypes/FLOAT
=== RUN TestTypeNamesAndScanTypes/DOUBLE
=== RUN TestTypeNamesAndScanTypes/TIMESTAMP
=== RUN TestTypeNamesAndScanTypes/DATE
=== RUN TestTypeNamesAndScanTypes/TIME
=== RUN TestTypeNamesAndScanTypes/INTERVAL
=== RUN TestTypeNamesAndScanTypes/HUGEINT
=== RUN TestTypeNamesAndScanTypes/VARCHAR
=== RUN TestTypeNamesAndScanTypes/BLOB
=== RUN TestTypeNamesAndScanTypes/DECIMAL(30,17)
=== RUN TestTypeNamesAndScanTypes/TIMESTAMP_S
=== RUN TestTypeNamesAndScanTypes/TIMESTAMP_MS
=== RUN TestTypeNamesAndScanTypes/TIMESTAMP_NS
=== RUN TestTypeNamesAndScanTypes/VARCHAR[][]
=== RUN TestTypeNamesAndScanTypes/STRUCT("key1"_VARCHAR,_"key2"_INTEGER,_"key3"_DOUBLE)
=== RUN TestTypeNamesAndScanTypes/STRUCT("key1_(,)_\_\""_""""_"""_INTEGER)
=== RUN TestTypeNamesAndScanTypes/MAP(INTEGER,_VARCHAR)
=== RUN TestTypeNamesAndScanTypes/UUID
--- PASS: TestTypeNamesAndScanTypes (0.02s)
--- PASS: TestTypeNamesAndScanTypes/BOOLEAN (0.00s)
--- PASS: TestTypeNamesAndScanTypes/TINYINT (0.00s)
--- PASS: TestTypeNamesAndScanTypes/SMALLINT (0.00s)
--- PASS: TestTypeNamesAndScanTypes/INTEGER (0.00s)
--- PASS: TestTypeNamesAndScanTypes/BIGINT (0.00s)
--- PASS: TestTypeNamesAndScanTypes/UTINYINT (0.00s)
--- PASS: TestTypeNamesAndScanTypes/USMALLINT (0.00s)
--- PASS: TestTypeNamesAndScanTypes/UINTEGER (0.00s)
--- PASS: TestTypeNamesAndScanTypes/UBIGINT (0.00s)
--- PASS: TestTypeNamesAndScanTypes/FLOAT (0.00s)
--- PASS: TestTypeNamesAndScanTypes/DOUBLE (0.00s)
--- PASS: TestTypeNamesAndScanTypes/TIMESTAMP (0.00s)
--- PASS: TestTypeNamesAndScanTypes/DATE (0.00s)
--- PASS: TestTypeNamesAndScanTypes/TIME (0.00s)
--- PASS: TestTypeNamesAndScanTypes/INTERVAL (0.00s)
--- PASS: TestTypeNamesAndScanTypes/HUGEINT (0.00s)
--- PASS: TestTypeNamesAndScanTypes/VARCHAR (0.00s)
--- PASS: TestTypeNamesAndScanTypes/BLOB (0.00s)
--- PASS: TestTypeNamesAndScanTypes/DECIMAL(30,17) (0.00s)
--- PASS: TestTypeNamesAndScanTypes/TIMESTAMP_S (0.00s)
--- PASS: TestTypeNamesAndScanTypes/TIMESTAMP_MS (0.00s)
--- PASS: TestTypeNamesAndScanTypes/TIMESTAMP_NS (0.00s)
--- PASS: TestTypeNamesAndScanTypes/VARCHAR[][] (0.00s)
--- PASS: TestTypeNamesAndScanTypes/STRUCT("key1"_VARCHAR,_"key2"_INTEGER,_"key3"_DOUBLE) (0.00s)
--- PASS: TestTypeNamesAndScanTypes/STRUCT("key1_(,)_\_\""_""""_"""_INTEGER) (0.00s)
--- PASS: TestTypeNamesAndScanTypes/MAP(INTEGER,_VARCHAR) (0.00s)
--- PASS: TestTypeNamesAndScanTypes/UUID (0.00s)
=== RUN TestMultipleStatements
--- PASS: TestMultipleStatements (0.01s)
=== RUN TestParquetExtension
--- PASS: TestParquetExtension (0.05s)
=== RUN TestPrepareQuery
--- PASS: TestPrepareQuery (0.01s)
=== RUN TestPrepareWithError
--- PASS: TestPrepareWithError (0.02s)
=== CONT TestOpen
=== CONT TestHugeInt
=== CONT TestBoolean
=== CONT TestBlob
=== CONT TestVarchar
=== CONT TestJSON
=== CONT TestTimestamp
=== CONT TestInterval
=== RUN TestOpen/without_config
=== RUN TestBlob/select_as_string
=== RUN TestInterval/bind_interval
=== RUN TestBlob/select_as_hex
=== RUN TestVarchar/size_is_smaller_than_12
=== RUN TestBoolean/scan
=== RUN TestInterval/scan_interval
=== RUN TestTimestamp/epoch
=== RUN TestInterval/scan_interval/simple_interval
=== RUN TestVarchar/size_is_exactly_12
=== RUN TestHugeInt/scan_hugeint
=== RUN TestHugeInt/scan_hugeint/0
=== RUN TestTimestamp/before_1970
=== RUN TestInterval/scan_interval/interval_arithmetic
=== RUN TestVarchar/size_is_13
=== RUN TestBoolean/bind
=== RUN TestHugeInt/scan_hugeint/1
=== RUN TestTimestamp/after_1970
=== RUN TestVarchar/size_is_greater_than_12
=== RUN TestInterval/scan_interval/timestamp_arithmetic
=== RUN TestJSON/select_empty_JSON
=== RUN TestHugeInt/scan_hugeint/-1
=== RUN TestTimestamp/HH:MM:SS
=== RUN TestJSON/select_from_marshalled_JSON
=== RUN TestTimestamp/HH:MM:SS.DDDD
=== RUN TestHugeInt/scan_hugeint/9223372036854775807
=== RUN TestHugeInt/scan_hugeint/-9223372036854775808
=== RUN TestHugeInt/scan_hugeint/170141183460469231731687303715884105727
=== RUN TestJSON/select_JSON_array
=== RUN TestHugeInt/scan_hugeint/-170141183460469231731687303715884105727
=== RUN TestHugeInt/bind_hugeint
--- PASS: TestBoolean (0.01s)
--- PASS: TestBoolean/scan (0.00s)
--- PASS: TestBoolean/bind (0.00s)
=== CONT TestEmpty
--- PASS: TestVarchar (0.02s)
--- PASS: TestVarchar/size_is_smaller_than_12 (0.00s)
--- PASS: TestVarchar/size_is_exactly_12 (0.00s)
--- PASS: TestVarchar/size_is_13 (0.00s)
--- PASS: TestVarchar/size_is_greater_than_12 (0.00s)
=== CONT TestDate
--- PASS: TestInterval (0.02s)
--- PASS: TestInterval/bind_interval (0.00s)
--- PASS: TestInterval/scan_interval (0.00s)
--- PASS: TestInterval/scan_interval/simple_interval (0.00s)
--- PASS: TestInterval/scan_interval/interval_arithmetic (0.00s)
--- PASS: TestInterval/scan_interval/timestamp_arithmetic (0.00s)
=== CONT TestUUID
=== RUN TestOpen/with_config
--- PASS: TestBlob (0.02s)
--- PASS: TestBlob/select_as_string (0.00s)
--- PASS: TestBlob/select_as_hex (0.00s)
=== CONT TestQuery
--- PASS: TestTimestamp (0.02s)
--- PASS: TestTimestamp/epoch (0.00s)
--- PASS: TestTimestamp/before_1970 (0.00s)
--- PASS: TestTimestamp/after_1970 (0.00s)
--- PASS: TestTimestamp/HH:MM:SS (0.00s)
--- PASS: TestTimestamp/HH:MM:SS.DDDD (0.00s)
=== CONT TestENUMs
--- PASS: TestJSON (0.02s)
--- PASS: TestJSON/select_empty_JSON (0.00s)
--- PASS: TestJSON/select_from_marshalled_JSON (0.00s)
--- PASS: TestJSON/select_JSON_array (0.00s)
=== CONT TestDecimal
--- PASS: TestHugeInt (0.02s)
--- PASS: TestHugeInt/scan_hugeint (0.00s)
--- PASS: TestHugeInt/scan_hugeint/0 (0.00s)
--- PASS: TestHugeInt/scan_hugeint/1 (0.00s)
--- PASS: TestHugeInt/scan_hugeint/-1 (0.00s)
--- PASS: TestHugeInt/scan_hugeint/9223372036854775807 (0.00s)
--- PASS: TestHugeInt/scan_hugeint/-9223372036854775808 (0.00s)
--- PASS: TestHugeInt/scan_hugeint/170141183460469231731687303715884105727 (0.00s)
--- PASS: TestHugeInt/scan_hugeint/-170141183460469231731687303715884105727 (0.00s)
--- PASS: TestHugeInt/bind_hugeint (0.00s)
=== CONT TestStruct
=== RUN TestDate/before_1970
=== RUN TestDate/after_1970
=== RUN TestDate/epoch
=== RUN TestQuery/simple
--- PASS: TestEmpty (0.01s)
=== RUN TestDecimal/decimal_widths
=== CONT TestList
=== RUN TestQuery/large_number_of_rows
--- PASS: TestDate (0.01s)
--- PASS: TestDate/before_1970 (0.00s)
--- PASS: TestDate/after_1970 (0.00s)
--- PASS: TestDate/epoch (0.00s)
=== CONT TestExec
=== RUN TestOpen/with_invalid_config
=== RUN TestStruct/scan_single_struct
--- PASS: TestOpen (0.03s)
--- PASS: TestOpen/without_config (0.02s)
--- PASS: TestOpen/with_config (0.01s)
--- PASS: TestOpen/with_invalid_config (0.00s)
=== CONT TestMap
=== RUN TestStruct/scan_slice_of_structs
--- PASS: TestUUID (0.02s)
--- PASS: TestENUMs (0.02s)
=== RUN TestList/integer_list
--- PASS: TestStruct (0.01s)
--- PASS: TestStruct/scan_single_struct (0.00s)
--- PASS: TestStruct/scan_slice_of_structs (0.00s)
=== RUN TestMap/select_map
=== RUN TestMap/select_nested_map
--- PASS: TestExec (0.01s)
=== RUN TestMap/insert_map
=== RUN TestDecimal/multiple_decimal_types
--- PASS: TestMap (0.01s)
--- PASS: TestMap/select_map (0.00s)
--- PASS: TestMap/select_nested_map (0.00s)
--- PASS: TestMap/insert_map (0.00s)
--- PASS: TestList (0.02s)
--- PASS: TestList/integer_list (0.01s)
=== RUN TestDecimal/huge_decimal
=== RUN TestDecimal/decimal_to_float64
--- PASS: TestDecimal (0.03s)
--- PASS: TestDecimal/decimal_widths (0.01s)
--- PASS: TestDecimal/multiple_decimal_types (0.00s)
--- PASS: TestDecimal/huge_decimal (0.00s)
--- PASS: TestDecimal/decimal_to_float64 (0.00s)
=== RUN TestQuery/wrong_syntax
=== RUN TestQuery/missing_parameter
=== RUN TestQuery/null
--- PASS: TestQuery (0.61s)
--- PASS: TestQuery/simple (0.00s)
--- PASS: TestQuery/large_number_of_rows (0.60s)
--- PASS: TestQuery/wrong_syntax (0.00s)
--- PASS: TestQuery/missing_parameter (0.00s)
--- PASS: TestQuery/null (0.00s)
PASS
ok github.com/marcboeker/go-duckdb 9.835sduckdb_test.go:45 comment out cause I did not compile this extension.ZenLiuCN
commented
10 months ago I suggest that maybe not merge the windows static link with go-duckdb for using mingw as compile now.
ZenLiuCN
commented
10 months ago Ps those extensions I compiled is under folder. And the extension compile script not finish yet.
ZenLiuCN
commented
10 months ago Ps, those three file are the script I use to compile the lib.
-----------------------------------------------------------------------
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 22/12/2023 16:29 490 build.sh
-a--- 22/12/2023 16:29 62 copyLib.sh
-a--- 22/12/2023 16:29 199 mergeStaticLib.shmergeStaticLib.sh not tested. just an Idea
ZenLiuCN
commented
10 months ago ilammy/msvc-dev-cmd seems should get a try. I will try on it (it may take some time)
ZenLiuCN
commented
10 months ago ilammy/msvc-dev-cmd seems should get a try. I will try on it (it may take some time)
done try. there still no stable method to link a static lib built with msvc .
ZenLiuCN
commented
10 months ago I moved relative codes to here, Cause the engine module is too big for pull from go mod server.
I will try to add a github workflow to build the mingw libs, maybe that will do a little help.
But also should notice that no official extensions provided for windows_amd64_mingw toolchain. Maybe requires extra provided extentions.
PS: I suggest that maybe put binaries to a seprated location or as release binary archives (so avoid to reach the go mod size limit , it'll broken everything, if with mingw lib and extensions it will reach the limit, maybe about 50mega bytes after zip compressed ) Merry Christmas ^_^
triiberg
commented
10 months ago -Wl,-Bstatic -lstdc++ -Wl,-Bstatic -lpthread
Hi. This very helpful, somehow it doesn't work in my computer. Do you have full build command? I think I'm missing some flags :( It might help others as well who does not have too much experience of compiling stuff.
ZenLiuCN
commented
10 months ago -Wl,-Bstatic -lstdc++ -Wl,-Bstatic -lpthread
Hi. This very helpful, somehow it doesn't work in my computer. Do you have full build command? I think I'm missing some flags :( It might help others as well who does not have too much experience of compiling stuff.
Here the the script for compile duckdb with mingw (GCC v13), You not need -Wl,-Bstatic -lstdc++ -Wl,-Bstatic -lpthread for compile. It's only required for link without dyn dlls.
#!/bin/sh
mkdir -p build && \
cmake -G "MinGW Makefiles" \
-DENABLE_EXTENSION_AUTOLOADING=1 \
-DENABLE_EXTENSION_AUTOINSTALL=1 \
-DDUCKDB_EXTENSION_CONFIGS="./.github/config/bundled_extensions.cmake" \
-DBUILD_SHELL=0 \
-DBUILD_BENCHMARK=0 \
-DBUILD_JDBC=0 \
-DBUILD_TPCH=0 \
-DBUILD_TPCDS=0 \
-DBUILD_ODBC=0 \
-DBUILD_PYTHON=0 \
-DDISABLE_UNITY=1 \
-DCMAKE_BUILD_TYPE=Release -B build &&\
cd build &&\
MAKEFLAGS=-j6 cmake --build . --config Release &&\
cd ../ &&\
mkdir -p libThe DuckDb part compiled very well. Thank you! I'm still don't understand how I can include it to my Go project's building process (like the simple Go project that initiates DuckDb database, writes one record). I need to create a Golang microservice using my linux, then compile it to other platforms: win and mac. It compiles to linux very well, no problem at all. But not for windows.
$ GOOS=windows GOARCH=amd64 CGO_ENABLED=1 CGO_LDFLAGS="-L./deps/windows_amd64_mingw" CGO_CFLAGS="-I./deps/includes" CC=x86_64-w64-mingw32-gcc CXX=x86_64-w64-mingw32-g++ go build -tags=duckdb_use_lib -o builds/app.exe cmd/main.go
returns result:
# command-line-arguments
/usr/local/go/pkg/tool/linux_amd64/link: running x86_64-w64-mingw32-g++ failed: exit status 1
/usr/bin/x86_64-w64-mingw32-ld: cannot find -lduckdb: No such file or directory
collect2: error: ld returned 1 exit statusand the same command but I remove the -tags=duckdb_use_lib I'll got this one:
# github.com/marcboeker/go-duckdb
In file included from duckdb.cpp:2:
duckdb.hpp:1686:12: error: ‘mutex’ has not been declared in ‘std’
1686 | using std::mutex;
| ^~~~~
In file included from duckdb.cpp:2:
duckdb.hpp:12582:2: error: ‘mutex’ does not name a type
12582 | mutex lock;
| ^~~~~
In file included from duckdb.cpp:2:
duckdb.hpp:14037:2: error: ‘mutex’ does not name a type
14037 | mutex lock;
| ^~~~~
duckdb.hpp:14679:2: error: ‘mutex’ does not name a type
14679 | mutex lock;
| ^~~~~
In file included from duckdb.cpp:2:
duckdb.hpp:16566:2: error: ‘mutex’ does not name a type
16566 | mutex producer_lock;
| ^~~~~
duckdb.hpp:16611:2: error: ‘mutex’ does not name a type
16611 | mutex thread_lock;
| ^~~~~
duckdb.hpp:16729:2: error: ‘mutex’ does not name a type
16729 | mutex batch_lock;
| ^~~~~
duckdb.hpp:16852:2: error: ‘mutex’ does not name a type
16852 | mutex executor_lock;
| ^~~~~
duckdb.hpp:16853:2: error: ‘mutex’ does not name a type
16853 | mutex error_lock;
| ^~~~~
duckdb.hpp:17907:2: error: ‘mutex’ does not name a type
17907 | mutex lock;
| ^~~~~
duckdb.hpp:18164:49: error: ‘mutex’ was not declared in this scope
18164 | typedef unordered_map<CatalogSet *, unique_lock<mutex>> set_lock_map_t;
| ^~~~~
duckdb.hpp:18164:49: error: template argument 1 is invalid
duckdb.hpp:18164:54: error: template argument 2 is invalid
18164 | typedef unordered_map<CatalogSet *, unique_lock<mutex>> set_lock_map_t;
| ^~
duckdb.hpp:18164:54: error: template argument 5 is invalid
duckdb.hpp:18277:72: error: ‘mutex’ was not declared in this scopeAnd if I put the .a files into project own directory (someone told me that -l option doesn't go for other directories?) I got errors like this:
# command-line-arguments
/usr/local/go/pkg/tool/linux_amd64/link: running x86_64-w64-mingw32-g++ failed: exit status 1
/usr/bin/x86_64-w64-mingw32-ld: /tmp/go-link-1052453679/000002.o:appender.cgo2.:(.text+0x8d): undefined reference to `__imp_duckdb_append_blob'
/usr/bin/x86_64-w64-mingw32-ld: /tmp/go-link-1052453679/000002.o:appender.cgo2.:(.text+0xe9): undefined reference to `__imp_duckdb_append_bool'
/usr/bin/x86_64-w64-mingw32-ld: /tmp/go-link-1052453679/000002.o:appender.cgo2.:(.text+0x147): undefined reference to `__imp_duckdb_append_double'
/usr/bin/x86_64-w64-mingw32-ld: /tmp/go-link-1052453679/000002.o:appender.cgo2.:(.text+0x1a4): undefined reference to `__imp_duckdb_append_float'
/usr/bin/x86_64-w64-mingw32-ld: /tmp/go-link-1052453679/000002.o:appender.cgo2.:(.text+0x200): undefined reference to `__imp_duckdb_append_int16'
/usr/bin/x86_64-w64-mingw32-ld: /tmp/go-link-1052453679/000002.o:appender.cgo2.:(.text+0x258): undefined reference to `__imp_duckdb_append_int32'That requires modify the go-duck source, I suggest use dynamic link for windows ( also ship the dll with yourbuilt go service) That will much easier. I doesn't tried cross-compile from linux to windows. See level 6 for details of modify go-duckdb. (Really not suggest use this solution for production , use dynamic linked will more stable, at least for windows)
camperjett
commented
4 months ago Hi! I am working on a go project on windows. How do I use this go-client?
ZenLiuCN
commented
4 months ago use dynamic lib link via tags. it is the most stable for current. unless U real need a standalone executable.
Is there any config or guidance for compiling on windows?