heap corruption (windows) at check(db.Ping())

[sdmcallister]

In regular Windows 10, I get exit status 0xc0000374 (heap corruption) at the line check(db.Ping()) when it panics. Running the example program.

Go 1.18.2, duckdb 0.3.4

Running with:

$env:LIB="duckdb.dll"
$env:CGO_LDFLAGS = "-LC:\Users\me\golang\duckdb_testing"
$env:CGO_CFLAGS = "-IC:\Users\me\golang\duckdb_testing"
go run -ldflags="-r C:\Users\me\golang\duckdb_testing" main.go

[marcboeker]

Meanwhile a lot has changed under the hood. Could you please check with the latest master and DuckDB 0.4.0 if you can reproduce this? Thanks! If it still occurs, do you have more debugging information?

[sdmcallister]

Thanks for the response.

Still having similar issue. I get more information if I write to disk using db, err := sql.Open("duckdb", "foobar.db")

Here is what I see:

PS C:\Users\sdmca\Desktop\src\golang\ddb> go run .
Opened!
2022/07/26 19:54:14 DB opened with access mode automatic
2022/07/26 19:54:14 marc is 99 years old, 1.91 tall, bday on 1970-01-01T00:00:00Z and has awesomeness: true
2022/07/26 19:54:14 macgyver is 70 years old, 1.85 tall, bday on 1951-01-23T00:00:00Z and has awesomeness: true
2022/07/26 19:54:14 Deleted 2 rows
2022/07/26 19:54:14 Starting transaction...
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x1 addr=0x20 pc=0x7ff712ad2633]

goroutine 1 [running]:
database/sql.(*DB).conn(0x0, {0x7ff712b6cc68, 0xc000012040}, 0x1)
        C:/Program Files/Go/src/database/sql/sql.go:1290 +0x53
database/sql.(*DB).begin(0x0?, {0x7ff712b6cc68, 0xc000012040}, 0xc000070050?, 0x30?)
        C:/Program Files/Go/src/database/sql/sql.go:1869 +0x33
database/sql.(*DB).BeginTx(0xc00011fd20?, {0x7ff712b6cc68, 0xc000012040}, 0x7ff712ac252a?)
        C:/Program Files/Go/src/database/sql/sql.go:1847 +0x7e
database/sql.(*DB).Begin(...)
        C:/Program Files/Go/src/database/sql/sql.go:1865
main.runTransaction()
        C:/Users/sdmca/Desktop/src/golang/ddb/main.go:91 +0x85
main.main()
        C:/Users/sdmca/Desktop/src/golang/ddb/main.go:78 +0x885
exit status 2

Running in-memory just:

PS C:\Users\sdmca\Desktop\src\golang\ddb> go run .
Opened!
exit status 0xc0000374

[marcboeker]

Thanks! I was able to reproduce this via GitHub actions (https://github.com/marcboeker/go-duckdb/runs/7548645334?check_suite_focus=true).

Now I need a way to get access to a Windows Vm where I can debug this in detail.

[rusco]

I tried again with the most up-to-date tech stack: Win11, Go 1.19.1, TDM-GCC, DuckDB 0.5.0 libraries and confirm the bug 😔 Commenting out the Ping() statements leads to the same memory corruption at the next db.Exec() statement instead.

These kind of bugs remember me to the early times of the https://github.com/mattn/go-sqlite3 bindings, at the beginning they happened only on windows.

[marcboeker]

There will be a release shortly with the ability to compile DuckDB from source and link it directly into the executable. If you want to test drive it, please see this PR.

[sdmcallister]

@marcboeker

Excited about this release.

Still having trouble on windows 10 though. Using the code from below:

package main

import (
        "database/sql"
        "log"
        "fmt"
        _ "github.com/marcboeker/go-duckdb"
)

var (
        db *sql.DB
)

func main() {
        var err error
        db, err = sql.Open("duckdb", "?access_mode=READ_WRITE")
        if err != nil {
                log.Fatal(err)
        }
        defer db.Close()

        if err := db.Ping; err != nil {
                log.Fatal(err)
        }
        fmt.Println("success")
}

In WSL (ubuntu) it builds:

I see the following error (which I just checked happens at the ping):

2022/10/02 19:55:41 0x8141e0

Trying to build in windows:

cc1plus.exe out of memory allocating 65536 bytes.

I found this. Maybe I'm paranoid, but is it weird I would need to mess with the boot config?

https://www.intel.com/content/www/us/en/support/programmable/articles/000086946.html

[marcboeker]

After spending several hours to debug this, I have found some reasons but no solution.

I can reproduce the problem with the exit status on Windows outside WSL bit in MSYS2 when using static linking using the duckdb.lib from their releases page. The file size is also extremely small compared to their dll. 1 mb vs 32 mb for the dll.

The out of memory is based on a 32 bit compile chain. Switching to 64 bits solves the OOM problem for me, but compilation does not finish. That is weird.

I'm tracking my progress in https://github.com/marcboeker/go-duckdb/issues/51 I have to get support from the DuckDB maintainers about the build process on windows.

[panta]

I've encountered the same problem cross-compiling for Windows. Suspecting an incompatibility with the Visual C++ based build of the official duckdb Windows libs, I've rebuild the duckdb dll using TDM-GCC 10.3.0 (using the the mingw-w64 based compiler) and then cross-compiled with duckdb_use_lib and it works without issues (even in a fairly complex application doing some complex duckdb queries). I've not tried doing the amalgamation cross-compile since it seems to need a lot of time on my machine.

[hw2499]

Trying to build in windows:

cc1plus.exe out of memory allocating 65536 bytes.

How to solve this problem?

[marcboeker]

I have too little experience with compiler chains on Windows and not even access to a native Windows machine. So any additional help is welcomed.