lunarmodules / luasql

LuaSQL is a simple interface from Lua to a DBMS.
http://lunarmodules.github.io/luasql
535 stars 192 forks source link

Accented characters with luasql-odbc #133

Open lodumont opened 2 years ago

lodumont commented 2 years ago

I am trying to retrieve data from a Filemaker database containing accented letters (such as é or è, with UTF 8 encoding) using luasql-odbc. For example, in the following code, I am trying to get a location name (here "Tréboul"), but the accented character takes two bytes and the last letter of the string is truncated.

driver = require "luasql.odbc"
env = driver.odbc()
cnn = env:connect("DB", "user", "pwd")
sql = "SELECT col1 FROM table WHERE col2 = 'xxx'"
stmt = cnn:execute(sql)
row = stmt:fetch({}, "a")

print(row.col1)
> Trébou

string.byte(row.col1, 1, -1)
> 84    114 195 169 98  111 117

As you can see the "é" letters is here composed of two bytes (195 and 169) and the final "l" is missing.

Here is what my odbc.ini file looks like (I'm using unixodbc):

[ODBC Data Sources]
DB = FileMaker ODBC

[DB]
Driver                 = /Library/ODBC/FileMaker ODBC.bundle/Contents/MacOS/fmodbc.so
Description            = DB
Server                 = localhost
CertificateFailureType = Warning
Database               = DB
UseLongVarchar         = 
AutoDetectEncoding     = 
WideAPI                = Yes
UnicodeTextTypes       = Yes
MultiByteEncoding      = UTF-8
QueryLog_On            = 
QueryLogTime           = 
QueryLogFile           = 
Charset                = UTF-8

I'm accessing the same database through ODBC with R (RODBC package) and I have no issue with these characters, so I thought this could come from luasql. Any idea what might solve this?