search

methew / pyodbc

Automatically exported from code.google.com/p/pyodbc
MIT No Attribution
0 stars 0 forks source link

UTF-16 in results instead of UTF-8, Mac OS X 64-bit #245

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. install pyodbc 3.0.3 on Python 2.7.2, Mac OS X 10.7.3 64-bit
2. build FreeTDS 0.91 with iODBC, make connection to MS SQL Server 2008
3. run a query against a table containing column "Text" type of nvarchar(200)

c = pyodbc.connect("DSN=myDsn;UID=myUser;PWD=myPwd;CHARSET=UTF8")
r = c.execute("select top 3 Text from mytable")
for row in r:
    print row

What is the expected output? What do you see instead?
- I got results in UTF-16, expected UTF-8. Problem looks very similar to 
http://code.google.com/p/pyodbc/issues/detail?id=78

What version of the product are you using? On what operating system?
- Mac OS X 10.7.3 64-bit
- Python 2.7.2
- pyodbc 3.0.3
- iODBC Driver Manager: 03.52.0607.1008 (this iODBC version is shipped by 
default together with Mac OS)
- FreeTDS 0.91
- MS SQL Server 2008

My config files:

odbc.ini:
---------------
[myDsn]
Description = myDb
Driver = TDS
Database = myDb
Servername = myServer
Uid = myUser
Pwd = myPwd

odbcinst.ini:
---------------
[global]
text size = 64512

[TDS]
Description = FreeTDS Driver
Driver = /usr/local/lib/libtdsodbc.so
UsageCount = 1

freetds.conf:
---------------
[myServer]
host = serverIP
port = 1433
tds version = 7.0
client charset = UTF-8

Please provide any additional information below.

I ran the same query against the same server, database and table using:
- Ubuntu 11.10 32-bit + Python 2.7.2 + pyodbc 3.0.3 + FreeTDS 0.91 + unixODBC 
2.3.1
- Windows 7 64-bit + Python 2.7.2 + pyodbc 3.0.2  + built-in ODBC + SQL 11 
Native Driver

both on Ubuntu and Windows results were returned in UTF-8. However when I ran 
the same script on Mac OS X 64-bit, I got results in UTF-16.

Original issue reported on code.google.com by denis.st...@unipag.com on 10 Mar 2012 at 7:26

GoogleCodeExporter commented 9 years ago 
The freetdstests.py all pass, and they test varchar and nvarchar for all of the 
following sizes: 0, 1, 255, 256, 510, 511, 512, 1023, 1024, 2047, 2048, 4000.

I'm guessing there is something else.

Can you retest with pyodbc 3.0.6.  I am also using tds version 7.2, though I'm 
not sure if that would be necessary.

If you are still getting the error, can you supply a small test (a couple of 
values and how you are checking that they are in UCS-16).

Original comment by mkleehammer on 27 Sep 2012 at 10:19

GoogleCodeExporter commented 9 years ago

Original comment by mkleehammer on 29 Sep 2012 at 4:59

GoogleCodeExporter commented 9 years ago 
I am seeing this on a 64bit mac with pyodbc 3.0.7 and unixodbc 2.3.2 connecting 
to vertica 6.1.2.  vsql and dbvisualizer both display rows in correct format.  
pyodbc in ipython notebook or python shell show lines like 
u'\U00103030\udc8d\udc35...'

using hombrew installed python 2.7.6 and anaconda python 2.7.6

odbc.ini (relevant parts)
[ODBC Data Sources]
Vertica = vertica dev server

[Vertica]
Driver = Vertica
DriverManagerEncoding=UTF-8

vertica.ini
[Driver]
DriverManagerEncoding=UTF-8
ODBCInstLib=/usr/local/lib/libodbcinst.dylib
ErrorMessagesPath=/opt/vertica/lib

Original comment by kesten.b...@gmail.com on 7 May 2014 at 6:05