search

YongpingZhang / pyodbc

Automatically exported from code.google.com/p/pyodbc
MIT No Attribution
0 stars 0 forks source link

FreeTDS->unixODBC->pyodbc character problem on linux #222

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1.I have a simple script that connect to a MSSQLSERVER 2008 db and extract the 
value of a VARCHAR field.
The db is encoded in SQL_Latin1_General_CP1_CL_AS.
My system is Ubuntu 11.04 and the python version:
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24) 
[GCC 4.5.2] on linux2
pyodbc version 2.1.11
The value contain a 'ì' character.
The goal is to export the value in an excel file with the xlwt package.
I run the same script on Windows too and the result is very different.
When I run on Windows (XP with python 2.5.4, pyodbc 2.1.6 driver={SQL Server}) 
I obtain a <type str> and to write in the file correctly I must use the unicode 
function: 
val = unicode(val,'Latin_1')
If I don't encode the string I obtain an error.
On linux with the same script I obtain always a '?' character, encoding or not.

2. To go in deep I did try the same extraction with the unixODBC directly using 
the 'isql' tool of the unixODBC distribution (I stress because I did find also 
a java tool with the same name...).
With isql I obtain the 'ì' character correctly.
So it seems the problem is between pyodbc and unixODBC...

3. I did try to analyse the str value extracted character by character.
The ord() function on windows return 236 for the 'ì' but on linux it return 63 
(I try to uderstand if the output '?' is a content or a rendering problem...)
So the character extracted is a '?'.

What is the expected output? What do you see instead?
'ì' -> 'ì' on Windows
'ì' -> '?' on Linux

What version of the product are you using? On what operating system?
WindowsXP with python 2.5.4, pyodbc 2.1.6 driver={SQL Server} is OK
Ubuntu 11.04 with python 2.7.1 pyodbc 2.1.11 driver={FreeTDS} NOT OK

Please provide any additional information below.
I did try also to cast the field at db level CAST(field, NVARCHAR) to estract 
directly a unicode value but it run ok on Windows (I don't have the necessity 
to encode the value with the unicode function) and nothing change on Linux.

Original issue reported on code.google.com by enricose...@hotmail.com on 3 Nov 2011 at 9:07

GoogleCodeExporter commented 9 years ago

Original comment by mkleehammer on 18 Dec 2011 at 6:54

GoogleCodeExporter commented 9 years ago 
This appears to be fixed in 3.0.x.  If you run the tests2/freetdstests.py unit 
tests, you'll see one named test_varchar_upperlatin that is equivalent.  (I 
also changed the character to 236 to match your test exactly and that worked 
also.)

Original comment by mkleehammer on 18 Dec 2011 at 7:03

GoogleCodeExporter commented 9 years ago 
Hi,

 I am facing a similar problem, using pyodbc '3.0.6'

 the database is reporting character iso_1 and collation Modern_Spanish_CI_AS 

 using isql I get the correct results, using pyodbc, I get ? where I sould be gettin ñ or accentuated vowels,

 best regards,
Alberto

Original comment by alberto....@gmail.com on 17 Oct 2012 at 3:50

GoogleCodeExporter commented 9 years ago 
python:  2.6.5 (r265:79063, Oct  1 2012, 22:04:36) 
[GCC 4.4.3]
pyodbc:  3.0.7-beta10 /usr/src/pyodbc/build/lib.linux-x86_64-2.6/pyodbc.so
odbc:    03.52
driver:  libtdsodbc.so 0.82
         supports ODBC version 03.00
os:      Linux
unicode: Py_Unicode=4 SQLWCHAR=2
server: MSSQL-2000, Cyrillic_General_BIN

tests2/freetdstests.py fails 36 tests with unicode, text, varchar, etc

Original comment by qwigly...@gmail.com on 11 Dec 2012 at 12:05

GoogleCodeExporter commented 9 years ago 
SQL_WCHAR is not supported by 0.82
http://lists.ibiblio.org/pipermail/freetds/2010q1/025572.html

Original comment by qwigly...@gmail.com on 11 Dec 2012 at 5:11