Big Panic when get null string value from db2 Server on mac

yyt030 commented 6 years ago

when i get the null string value, execute the go source code, happen panic,

log:


=== RUN   TestNullString
--- FAIL: TestNullString (0.11s)
panic: runtime error: slice bounds out of range [recovered]
panic: runtime error: slice bounds out of range

goroutine 5 [running]: testing.tRunner.func1(0xc4200b60f0) /usr/local/Cellar/go/1.10.2/libexec/src/testing/testing.go:742 +0x29d panic(0x41383e0, 0x4223250) /usr/local/Cellar/go/1.10.2/libexec/src/runtime/panic.go:502 +0x229 go-db2-example/vendor/github.com/alexbrainman/odbc.(BindableColumn).Value(0xc420010880, 0x500da00, 0x0, 0xc420043e30, 0x4040ef7, 0x10, 0x41380e0) /Users/xxx/workspaces/go/src/go-db2-example/vendor/github.com/alexbrainman/odbc/column.go:247 +0x305 go-db2-example/vendor/github.com/alexbrainman/odbc.(Rows).Next(0xc42000e058, 0xc4200465c0, 0x1, 0x1, 0x1, 0x1) /Users/xxx/workspaces/go/src/go-db2-example/vendor/github.com/alexbrainman/odbc/rows.go:35 +0xbf database/sql.(Rows).nextLocked(0xc4200a6100, 0xc420010000) /usr/local/Cellar/go/1.10.2/libexec/src/database/sql/sql.go:2622 +0xc4 database/sql.(Rows).Next.func1() /usr/local/Cellar/go/1.10.2/libexec/src/database/sql/sql.go:2600 +0x3c database/sql.withLock(0x417f660, 0xc4200a6130, 0xc420043f00) /usr/local/Cellar/go/1.10.2/libexec/src/database/sql/sql.go:3032 +0x63 database/sql.(Rows).Next(0xc4200a6100, 0x416d298) /usr/local/Cellar/go/1.10.2/libexec/src/database/sql/sql.go:2599 +0x7a go-db2-example.TestNullString(0xc4200b60f0) /Users/xxx/workspaces/go/src/go-db2-example/go_db2_test.go:225 +0x12e testing.tRunner(0xc4200b60f0, 0x416d378) /usr/local/Cellar/go/1.10.2/libexec/src/testing/testing.go:777 +0xd0 created by testing.(T).Run /usr/local/Cellar/go/1.10.2/libexec/src/testing/testing.go:824 +0x2e0

- source code:

import ( "database/sql" "log" "testing"

"github.com/alexbrainman/odbc"
_ "github.com/alexbrainman/odbc"

)

// Some code is Omitted ...

func TestNullString(t *testing.T) { db, _ := NewConn() defer db.Close()

rows, err := db.Query("select name from test")
if err != nil {
    t.Fatalf("error:%v", err)
}
defer rows.Close()

for rows.Next() {

}

}

- Env:

GOARCH="amd64" GOBIN="" GOCACHE="/Users/user/Library/Caches/go-build" GOEXE="" GOHOSTARCH="amd64" GOHOSTOS="darwin" GOOS="darwin" GOPATH="/Users/user/workspaces/go" GORACE="" GOROOT="/usr/local/Cellar/go/1.10.2/libexec" GOTMPDIR="" GOTOOLDIR="/usr/local/Cellar/go/1.10.2/libexec/pkg/tool/darwin_amd64" GCCGO="gccgo" CC="clang" CXX="clang++" CGO_ENABLED="1" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/pb/r1fk_rw15q7d7v2cj9d4v_440000gn/T/go-build475294692=/tmp/go-build -gno-record-gcc-switches -fno-common"

- database:

DB21085I This instance or install (instance name, where applicable: "db2inst1") uses "64" bits and DB2 code release "SQL10055" with level identifier "0606010E". Informational tokens are "DB2 v10.5.0.5", "s141128", "IP23633", and Fix Pack "5". Product is installed at "/home/db2inst1/sqllib".

- table

                            Data type                     Column

Column name schema Data type name Length Scale Nulls

ID SYSIBM INTEGER 4 0 Yes
NAME SYSIBM VARCHAR 255 0 Yes


- table value 
the Column Name is null value;

-  Debug?
when i debug the function, and see the alexbrainman/odbc.(*BindableColumn).Value  in column.go:line 233, the function return "return c.BaseColumn.Value(c.Buffer[:c.Len])"
but i found the c.Len is Big Big value, looks not initially or point over
is it a BUG ? pls help me  

thanks

alexbrainman commented 6 years ago

Unfortunately I do not have any good suggestions on how to debug this. This driver was written to access my MS SQL Server, so it might have some MS SQL Server specific assumptions. You would have to try and print all variables as code run, and try to understand the code and see where code assumptions do not match to what your driver returns.

I could try and debug it here, but I would need to reproduce your problem here. So I would need a database server and I would need to install appropriate client software. I have small Linux computer. Do you think it is good enough to stage your environment?

Alex

yyt030 commented 6 years ago

@ Alex, i am not too familiar with cgo, so I have not found the reason for the error. you can pull ibmcom/db2express-c docker image as database server, download db2 ODBC client from the url https://public.dhe.ibm.com/ibmdl/export/pub/software/data/db2/drivers/odbc_cli/macos64_odbc_cli.tar.gz for linux is https://public.dhe.ibm.com/ibmdl/export/pub/software/data/db2/drivers/odbc_cli/linuxx64_odbc_cli.tar.gz try it and test , is it failed ?pls tell me.

alexbrainman commented 6 years ago

for linux is https://public.dhe.ibm.com/ibmdl/export/pub/software/data/db2/drivers/odbc_cli/linuxx64_odbc_cli.tar.gz try it and test

I will try. Do they have instructions on how to install that?

Alex

alexbrainman commented 6 years ago

I will try. Do they have instructions on how to install that?

@yyt030 I quickly looked at contents of https://public.dhe.ibm.com/ibmdl/export/pub/software/data/db2/drivers/odbc_cli/linuxx64_odbc_cli.tar.gz and it does not have any instructions on how to install that. I cannot help you with the product built by someone else if there are no instructions. Sorry.

Alex

yyt030 commented 6 years ago

sorry i am late for reply you. now i am outside and access web site by phone，the rate is too slow😊 please follow the instructction：github.com/yyt030/go-db2-example . its chinese，you tell me when you have any trouble

thanks

yyt030 commented 6 years ago

how to config db2cli.ini ：http://www.unixodbc.org/doc/db2.html

btw，the db2 odbc driver not need install，you need unpack gz file into any directory ，then set driver path as that url

@alexbrainman pls refer this url : https://www.easysoft.com/products/data_access/odbc-db2-driver/unixodbc.html

yyt030 commented 6 years ago

@alexbrainman, I try debug this and find something. when goto the Next function in github.com/alexbrainman/odbc/rows.go

func (r *Rows) Next(dest []driver.Value) error {
    ret := api.SQLFetch(r.os.h)
    if ret == api.SQL_NO_DATA {
        return io.EOF
    }
    if IsError(ret) {
        return NewError("SQLFetch", r.os.h)
    }
    for i := range dest {
        v, err := r.os.Cols[i].Value(r.os.h, i)
        if err != nil {
            return err
        }
        dest[i] = v
    }
    return nil
}

when execute api.SQLFetch(r.os.h), ret = 0 ,but the r.os.Cols[0] is below:

BaseColumn = {*go-db2-example/vendor/github.com/alexbrainman/odbc.BaseColumn} 
IsBound = true
IsVariableWidth = true
Size = 255
Len = 4294967295
Buffer = {[]uint8} len:255, cap:255

when goto into function in github.com/alexbrainman/odbc/column.go:

func (c *BindableColumn) Value(h api.SQLHSTMT, idx int) (driver.Value, error) {
    if !c.IsBound {
        ret := c.Len.GetData(h, idx, c.CType, c.Buffer)
        if IsError(ret) {
            return nil, NewError("SQLGetData", h)
        }
    }

    if c.Len.IsNull() {
        // is NULL
        return nil, nil
    }
    if !c.IsVariableWidth && int(c.Len) != c.Size {
        return nil, fmt.Errorf("wrong column #%d length %d returned, %d expected", idx, c.Len, c.Size)
    }
    return c.BaseColumn.Value(c.Buffer[:c.Len])
}

it got Panic when return slice c.Buffer[:c.Len]. the reason is c.Len = 4294967295, c.Len is 4 byte, max value is 0xffffffff = 4294967295 the bug is Similar to #100 what i do next ?

alexbrainman commented 6 years ago

how to ...

Thank you for providing the references.

the db2 odbc driver not need install，you need unpack gz file into any directory ，then set driver path as that url

I suspect you did not configure your system properly. You must use correct 32-bit or 64-bit version of your driver. The version must match version of Go that you use.

If you run go version, the output will tell you what Go version you use. Use correspondent 32-bit or 64-bit version of your ODBC driver. Please not, that referring to correct driver file in odbcinst.ini and odbc.ini is not enough. When Go builds your program, it also uses some C include files that have to be configured properly, otherwise C compiler calls your ODBC driver functions with wrong arguments. https://www.easysoft.com/products/data_access/odbc-db2-driver/unixodbc.html provides some details:

64-bit Platforms If your application is 64-bit, you need to use a 64-bit DB2 ODBC driver and build a 64-bit version of the unixODBC driver manager. (If you are not sure whether your application is 64-bit, run the file command on the application binary. If the file is 64-bit, the command output will contain "ELF 64-bit" (or something similar such as "ELF-Class64" or "ELF-64".) In unixODBC 2.2.13, the default size of SQLLEN/SQLULEN types changed from 32 bits to 64 bits. This is the expected size for the 64-bit DB2 ODBC driver, which is built with 64-bit SQLLEN types.

It is OK for you not to follow instructions and install software with ... unpack gz file into any directory ，then set driver path as thaturl ..., but then you have to fix broken things yourself.

Also see issue #100 - I think it is very similar to yours.

I hope it helps.

Alex

yyt030 commented 6 years ago

Thanks a lot, I will try this later, and feedback to you!

yyt030 commented 6 years ago

@alexbrainman , I confirm it again and that the env is not wrong match .

go env: it's amd64

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/xxx/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/xxx/workspaces/go"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.10.2/libexec"

unixODBC config file


$ cat /Users/xxx/.odbc.ini
[DB2_SAMPLE]
Driver=/Users/xxx/Downloads/clidriver/lib/libdb2.dylib
Description=Sample DB2 ODBC Database

$ file /Users/xxx/Downloads/clidriver/lib/libdb2.dylib /Users/xxx/Downloads/clidriver/lib/libdb2.dylib: Mach-O 64-bit dynamically linked shared library x86_64

yes, the load db2 ODBC driver libdb2.dylib is 64-bit.
- check unixODBC version

$ brew info unixODBC unixODBC: stable 2.3.6 (bottled) ODBC 3 connectivity for UNIX http://www.unixodbc.org/ Conflicts with: virtuoso (because Both install isql binaries.) /usr/local/Cellar/unixODBC/2.3.6 (46 files, 1.8MB) Poured from bottle on 2018-05-23 at 23:18:36 From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/unixodbc.rb ==> Dependencies Required: libtool ✔

as above write the 2.3.6 default is 64-bit and define C.SQLLEN is int64 on mac.

Then I use isql command tool which is the unixODBC package included.

$ isql DB2_SAMPLE db2inst1 123456 +---------------------------------------+	Connected!
sql-statement
help [tablename]
quit

+---------------------------------------+ SQL> select name,id from test; +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+ | NAME | ID | +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+ | | 1 | +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+ SQLRowCount returns -1 1 rows fetched

as above, the name is null and show nothing;

- use another package:  github.com/weigj/go-odbc

package main

import ( "fmt"

"github.com/weigj/go-odbc"

)

func main() { conn, err := odbc.Connect("DSN=DB2_SAMPLE;UID=db2inst1;PWD=123456") if err != nil { panic(err) } defer conn.Close()

stmt, err := conn.ExecDirect("select id, name from test")
if err != nil {
    panic(err)
}
defer stmt.Close()

rows, err := stmt.FetchAll()
if err != nil {
    panic(err)
}

for _, row := range rows {
    fmt.Printf("id:[%d], name:[%v]\n", row.Data[0], row.Data[1])
}

} // output: // id:[1], name:[]


Do you have other suggestions?

thanks

yyt030 commented 6 years ago

@alexbrainman , I rebuild my env. on virtualbox guest os , which is archlinux x86_64/db2express-c/unixODBC, the db2 odbc driver that is from db2 express-c package. That is OK!

As above, I think the panic on mac the reason is db2 odbc driver for mac! And the conclution is done after I download new package,v11.1.1fp1a_macos_dsdriver.dmg, replace

Driver=/Users/xxx/Downloads/clidriver/lib/libdb2.dylib

with

Driver=/Users/xxx/Downloads/clidriver/lib/libdb2o.dylib

the libdb2o.dylib is match for my env! the libdb2o.dylib not include in old package.

close the issue!

thanks!

alexbrainman commented 6 years ago

@yyt030 I am glad it worked for you.

Closing this issue.

Alex

alexbrainman / odbc

Big Panic when get null string value from db2 Server on mac #113