search

SAP / go-ase

SAP ASE Database Client for Go
Apache License 2.0
25 stars 14 forks source link

Handle LONGCHAR datatype #46

Closed apoorvdeshmukh closed 4 years ago

apoorvdeshmukh commented 4 years ago

This commit handles the LONGCHAR datatype by treating it same as CHAR and VARCHAR. This fixes the issue #45.

Issue: #45

Description

There was an issue with LONGCHAR data type as it wasn't handled. Following error was seen during 'make examples'

Opening database
Creating table 'simple'
Writing a=2147483647, b='a string' to table
Querying values from table
Displaying results of query
| a          | b                              |
2020/05/25 09:46:34 Failed: Error reading rows: Unhandled Go type: LONGCHAR
exit status 1
make: *** [Makefile:24: examples/cgo/simple] Error 1

How was the patch tested?

The patch was tested with the existing examples and all examples could be run successfully.

$:~/opensource_repo/go-ase> make examples
Running example: examples/cgo/simple
go run ./examples/cgo/simple/main.go
# github.com/SAP/go-ase/cgo
bridge.c: In function ‘ct_callback_server_message’:
bridge.c:6:9: warning: implicit declaration of function ‘srvMsg’ [-Wimplicit-function-declaration]
  return srvMsg(msg);
         ^~~~~~
bridge.c: In function ‘ct_callback_client_message’:
bridge.c:10:9: warning: implicit declaration of function ‘cltMsg’ [-Wimplicit-function-declaration]
  return cltMsg(msg);
         ^~~~~~
Opening database
Creating table 'simple'
Writing a=2147483647, b='a string' to table
Querying values from table
Displaying results of query
| a          | b                              |
| 2147483647 | a string                       |
Running example: examples/cgo/recorder
go run ./examples/cgo/recorder/main.go
# github.com/SAP/go-ase/cgo
bridge.c: In function ‘ct_callback_server_message’:
bridge.c:6:9: warning: implicit declaration of function ‘srvMsg’ [-Wimplicit-function-declaration]
  return srvMsg(msg);
         ^~~~~~
bridge.c: In function ‘ct_callback_client_message’:
bridge.c:10:9: warning: implicit declaration of function ‘cltMsg’ [-Wimplicit-function-declaration]
  return cltMsg(msg);
         ^~~~~~
Opening database
Creating MessageRecorder
Registering handler with server message broker
Enable traceflag 3604
Received messages:
DBCC execution completed. If DBCC printed error messages, contact a user with System Administrator (SA) role.
Listing enabled traceflags
Received messages:
Active traceflags: 1623, 3604

DBCC execution completed. If DBCC printed error messages, contact a user with System Administrator (SA) role.

Integration Tests:

I302026@linux-jtio:~/opensource_repo/go-ase> make integration
go test ./tests/cgotest
# github.com/SAP/go-ase/cgo
bridge.c: In function ‘ct_callback_server_message’:
bridge.c:6:9: warning: implicit declaration of function ‘srvMsg’ [-Wimplicit-function-declaration]
  return srvMsg(msg);
         ^~~~~~
bridge.c: In function ‘ct_callback_client_message’:
bridge.c:10:9: warning: implicit declaration of function ‘cltMsg’ [-Wimplicit-function-declaration]
  return cltMsg(msg);
         ^~~~~~
ok      github.com/SAP/go-ase/tests/cgotest (cached)
go test ./examples/cgo/...
# github.com/SAP/go-ase/cgo
bridge.c: In function ‘ct_callback_server_message’:
bridge.c:6:9: warning: implicit declaration of function ‘srvMsg’ [-Wimplicit-function-declaration]
  return srvMsg(msg);
         ^~~~~~
bridge.c: In function ‘ct_callback_client_message’:
bridge.c:10:9: warning: implicit declaration of function ‘cltMsg’ [-Wimplicit-function-declaration]
  return cltMsg(msg);
         ^~~~~~
ok      github.com/SAP/go-ase/examples/cgo/recorder (cached)
ok      github.com/SAP/go-ase/examples/cgo/simple   (cached)
CLAassistant commented 4 years ago

CLA assistant check
All committers have signed the CLA.

ntnn commented 4 years ago

Thanks! Please also add samples to tests/libtest/samples.go and the invocation of the generated test to tests/cgotest/main_test.go. To generate the tests please run go generate ./tests/... and verify the code and tests are working with make integration-cgo in your dev environment (LD_LIBRARY_PATH and SYBASE set).

apoorvdeshmukh commented 4 years ago

Thanks! Please also add samples to tests/libtest/samples.go and the invocation of the generated test to tests/cgotest/main_test.go. To generate the tests please run go generate ./tests/... and verify the code and tests are working with make integration-cgo in your dev environment (LD_LIBRARY_PATH and SYBASE set).

Thank you for the comments! ASE does not have a column type longchar and I noticed that the test in tests/libtest/samples.go have a corresponding ASE column type. This issue is happening against a column of def char(30) and there are already tests for this particular column type. However, I tried adding tests with char(30) column and repro string but it seems `make integration-cgo' is broken in it's existing state with ASE 16.0 SP03 PL07

apoorvd@linux-jtio:~/repo/go-ase> make integration-cgo
go test ./tests/cgotest
# github.com/SAP/go-ase/cgo
bridge.c: In function ‘ct_callback_server_message’:
bridge.c:6:9: warning: implicit declaration of function ‘srvMsg’ [-Wimplicit-function-declaration]
  return srvMsg(msg);
         ^~~~~~
bridge.c: In function ‘ct_callback_client_message’:
bridge.c:10:9: warning: implicit declaration of function ‘cltMsg’ [-Wimplicit-function-declaration]
  return cltMsg(msg);
         ^~~~~~
2020/05/31 15:57:14 Failed to setup simple DSN: Failed to setup database: Failed to create database: Received error reading results: Command failed, cancelled: Unknown error code: 1
FAIL    github.com/SAP/go-ase/tests/cgotest 3.549s
FAIL
make: *** [Makefile:12: integration-cgo] Error 1

Can you tell me what could be the issue here?

ntnn commented 4 years ago

Do you have a default device set up for new databases? The tests don't select a device but rather expect a default device set up already.

It's weird that ASE doesn't support LONGCHAR - the type is defined here in cstypes.h: https://github.com/SAP/go-ase/blob/master/cgo/includes/cstypes.h#L293 Although that may only be a definition for IQ, as Client-Library is used for both IQ and ASE. But than again it's weird that it reports LONGCHAR for you, as the columntype is used directly as reported by ASE with no transformation or detection done by go-ase.

That's why my first though was that your ASE may have an auto conversion from char(x) to longchar or similar (if ASE supports something like that).

Could you also post which ASE and Client-Library version you use? Maybe I can replicate the behaviour.

You could also check if the examples show the same error with LONGCHAR when using this branch: https://github.com/ntnn/go-ase/tree/current_state

apoorvdeshmukh commented 4 years ago

By default, master will be considered as default device for ASE if nothing else is set explicitly. There was an issue with my ASE server. (The default device 'master' needed more space to create the new database) After resolving this issue I got following set of errors.

I302026@linux-jtio:~/opensource_repo/go-ase> make integration
go test ./tests/cgotest
# github.com/SAP/go-ase/cgo
bridge.c: In function ‘ct_callback_server_message’:
bridge.c:6:9: warning: implicit declaration of function ‘srvMsg’ [-Wimplicit-function-declaration]
  return srvMsg(msg);
         ^~~~~~
bridge.c: In function ‘ct_callback_client_message’:
bridge.c:10:9: warning: implicit declaration of function ‘cltMsg’ [-Wimplicit-function-declaration]
  return cltMsg(msg);
         ^~~~~~
2020/06/01 10:50:55 Callback: ct_dynamic(): user api layer: external error: This routine cannot be called when a command has been initialized but not sent.
2020/06/01 10:50:55 Callback: ct_dynamic(): user api layer: external error: This routine cannot be called when a command has been initialized but not sent.
2020/06/01 10:50:55 Callback: ct_dynamic(): user api layer: external error: This routine cannot be called when a command has been initialized but not sent.
2020/06/01 10:50:55 Callback: ct_dynamic(): user api layer: external error: This routine cannot be called when a command has been initialized but not sent.
--- FAIL: TestVarChar (0.54s)
    --- FAIL: TestVarChar/username_password (0.10s)
        type_varchar.go:29: Error preparing table: Failed to execute prepared statement with : Unhandled column type: LONGCHAR
    --- FAIL: TestVarChar/username_password_connector (0.12s)
        type_varchar.go:29: Error preparing table: Failed to execute prepared statement with : Unhandled column type: LONGCHAR
    --- FAIL: TestVarChar/userstorekey (0.09s)
        type_varchar.go:29: Error preparing table: Failed to execute prepared statement with : Unhandled column type: LONGCHAR
    --- FAIL: TestVarChar/userstorekey_connector (0.10s)
        type_varchar.go:29: Error preparing table: Failed to execute prepared statement with : Unhandled column type: LONGCHAR
2020/06/01 10:50:55 Callback: ct_dynamic(): user api layer: external error: This routine cannot be called when a command has been initialized but not sent.
2020/06/01 10:50:55 Callback: ct_dynamic(): user api layer: external error: This routine cannot be called when a command has been initialized but not sent.
2020/06/01 10:50:55 Callback: ct_dynamic(): user api layer: external error: This routine cannot be called when a command has been initialized but not sent.
2020/06/01 10:50:56 Callback: ct_dynamic(): user api layer: external error: This routine cannot be called when a command has been initialized but not sent.
--- FAIL: TestChar (0.54s)
    --- FAIL: TestChar/username_password (0.08s)
        type_char.go:29: Error preparing table: Failed to execute prepared statement with : Unhandled column type: LONGCHAR
    --- FAIL: TestChar/username_password_connector (0.11s)
        type_char.go:29: Error preparing table: Failed to execute prepared statement with : Unhandled column type: LONGCHAR
    --- FAIL: TestChar/userstorekey (0.09s)
        type_char.go:29: Error preparing table: Failed to execute prepared statement with : Unhandled column type: LONGCHAR
    --- FAIL: TestChar/userstorekey_connector (0.13s)
        type_char.go:29: Error preparing table: Failed to execute prepared statement with : Unhandled column type: LONGCHAR
2020/06/01 10:50:56 Callback: ct_dynamic(): user api layer: external error: This routine cannot be called when a command has been initialized but not sent.
2020/06/01 10:50:56 Callback: ct_dynamic(): user api layer: external error: This routine cannot be called when a command has been initialized but not sent.
2020/06/01 10:50:56 Callback: ct_dynamic(): user api layer: external error: This routine cannot be called when a command has been initialized but not sent.
2020/06/01 10:50:56 Callback: ct_dynamic(): user api layer: external error: This routine cannot be called when a command has been initialized but not sent.
--- FAIL: TestNChar (0.50s)
    --- FAIL: TestNChar/username_password (0.10s)
        type_nchar.go:29: Error preparing table: Failed to execute prepared statement with : Unhandled column type: LONGCHAR
    --- FAIL: TestNChar/username_password_connector (0.11s)
        type_nchar.go:29: Error preparing table: Failed to execute prepared statement with : Unhandled column type: LONGCHAR
    --- FAIL: TestNChar/userstorekey (0.11s)
        type_nchar.go:29: Error preparing table: Failed to execute prepared statement with : Unhandled column type: LONGCHAR
    --- FAIL: TestNChar/userstorekey_connector (0.08s)
        type_nchar.go:29: Error preparing table: Failed to execute prepared statement with : Unhandled column type: LONGCHAR
2020/06/01 10:50:56 Callback: ct_dynamic(): user api layer: external error: This routine cannot be called when a command has been initialized but not sent.
2020/06/01 10:50:56 Callback: ct_dynamic(): user api layer: external error: This routine cannot be called when a command has been initialized but not sent.
2020/06/01 10:50:56 Callback: ct_dynamic(): user api layer: external error: This routine cannot be called when a command has been initialized but not sent.
2020/06/01 10:50:57 Callback: ct_dynamic(): user api layer: external error: This routine cannot be called when a command has been initialized but not sent.
--- FAIL: TestNVarChar (0.48s)
    --- FAIL: TestNVarChar/username_password (0.08s)
        type_nvarchar.go:29: Error preparing table: Failed to execute prepared statement with : Unhandled column type: LONGCHAR
    --- FAIL: TestNVarChar/username_password_connector (0.08s)
        type_nvarchar.go:29: Error preparing table: Failed to execute prepared statement with : Unhandled column type: LONGCHAR
    --- FAIL: TestNVarChar/userstorekey (0.10s)
        type_nvarchar.go:29: Error preparing table: Failed to execute prepared statement with : Unhandled column type: LONGCHAR
    --- FAIL: TestNVarChar/userstorekey_connector (0.11s)
        type_nvarchar.go:29: Error preparing table: Failed to execute prepared statement with : Unhandled column type: LONGCHAR
FAIL
2020/06/01 10:51:03 Tests failed with 1
FAIL    github.com/SAP/go-ase/tests/cgotest 25.828s
FAIL
make: *** [Makefile:12: integration-cgo] Error 1

This, I resolved by treating LONGCHAR same as case of CHAR in statements.go:304 after this, the integration tests could pass I will update this change in my PR.

I302026@linux-jtio:~/opensource_repo/go-ase> make integration
go test ./tests/cgotest
# github.com/SAP/go-ase/cgo
bridge.c: In function ‘ct_callback_server_message’:
bridge.c:6:9: warning: implicit declaration of function ‘srvMsg’ [-Wimplicit-function-declaration]
  return srvMsg(msg);
         ^~~~~~
bridge.c: In function ‘ct_callback_client_message’:
bridge.c:10:9: warning: implicit declaration of function ‘cltMsg’ [-Wimplicit-function-declaration]
  return cltMsg(msg);
         ^~~~~~
ok      github.com/SAP/go-ase/tests/cgotest (cached)
go test ./examples/cgo/...
# github.com/SAP/go-ase/cgo
bridge.c: In function ‘ct_callback_server_message’:
bridge.c:6:9: warning: implicit declaration of function ‘srvMsg’ [-Wimplicit-function-declaration]
  return srvMsg(msg);
         ^~~~~~
bridge.c: In function ‘ct_callback_client_message’:
bridge.c:10:9: warning: implicit declaration of function ‘cltMsg’ [-Wimplicit-function-declaration]
  return cltMsg(msg);
         ^~~~~~
ok      github.com/SAP/go-ase/examples/cgo/recorder (cached)
ok      github.com/SAP/go-ase/examples/cgo/simple   (cached)

Regarding longchar, there is more information here http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc32840.1570100/html/ctref/ctref366.htm

ntnn commented 4 years ago

Thanks, it's weird that ASE/Client-Library is reporting longchar when neither the column is a longchar and when longchar isn't even supported by ASE.

Please post the version of your ASE and Client-Library so I can replicate and isolate the issue - thanks!

apoorvdeshmukh commented 4 years ago

ASE Version: Adaptive Server Enterprise/16.0 SP03 PL08/EBF XXXXX SMP/P/x86_64/SLES 11.1/ase1 60sp03plx/3634/64-bit/FBO/Wed Sep 25 17:01:26 2019 Client Library Version: SAP Client-Library/16.0 SP03 PL07/P-EBF28824/DRV.16.0.03.07/Linux x86_64/Linux 2.6.18-128.el5 x86_64/BUILD1600-024/64bit/OPT/Wed Apr 24 02:05:34 2019

ntnn commented 4 years ago

@apoorvdeshmukh Please run the examples from the longchar_fix branch in my repo: https://github.com/ntnn/go-ase/tree/longchar_fix

I've updated your commit there so that LONGCHAR uses the same codepath as TEXT for writing, as LONGCHAR supports more characters than CHAR, resulting in written strings being cut off after CS_MAX_CHAR (256 characters).

I wasn't able to reproduce ASE reporting LONGCHAR for the CHAR column using the same version von ASE and Client-Library as you. Do you have any additional configuration in your installation and how do you install the ASE instance?

Thanks

ntnn commented 4 years ago

Closing, changes are included in #52.