Encoding of Germanic umlauts with Firebird DB

[GoogleCodeExporter]

I dont't know if this is a general issue, but if I connect my go app via ODBC 
to a Firebird database and try to marshal the selected output, I get the error 
'json: invalid UTF-8 in string:' if the string contains a Germanic umlaut (e.g. 
ä, ü, ö, ß). The character set of the connection doesn't matter ('None' and 
'UTF8').

I'm using a Firebird 2.5 64bit server (SuperClassic) with ODBC driver version 
2.0.0.01.152 on Windows 7 64bit.

The following works for me. As desciped on Stackoverflow 
(http://stackoverflow.com/questions/13510458/golang-convert-iso8859-1-to-utf8), 
I have to convert the variable 'buf' in line 116 of column.go to UTF-8.

As mentioned before, I dont't know if it's only my problem. If not, maybe this 
will help others.

Original issue reported on code.google.com by michael....@googlemail.com on 17 Jun 2013 at 10:04

[GoogleCodeExporter]

Please provide small program to demonstrate the issue. I will see if I can 
install the database to debug. Thank you.

Alex

Original comment by alex.bra...@gmail.com on 17 Jun 2013 at 10:57

[GoogleCodeExporter]

Thanks for the reply! Here is my minimal program:

http://play.golang.org/p/uvQj31j-Jl

And here are the other components I use (everything 64bit):

http://www.firebirdsql.org/en/firebird-2-5-2-upd1/
http://www.firebirdsql.org/en/odbc-driver/

The database field is e.g. VARCHAR(256). The odbc connection uses the default 
values (except DB user and pw).

Original comment by michael....@googlemail.com on 17 Jun 2013 at 11:54

[GoogleCodeExporter]

michael.rauh77,

I don't know anything about Firebird DB, but, I suspect, you cannot store 
non-ascii data in VARCHAR(256) without any special consideration. I read this 
http://www.slideshare.net/mindthebird/character-sets-and-unicode-in-firebird, 
and it gave me an idea to have "create table temp (alex varchar(256) character 
set utf8)". This did the trick and I was able to store and retrive back utf8 
strings with Go and my driver.

I am leaving it up to you to figure how you are going to use it. I am not sure 
what else I can do for you. Sorry.

Alex

Original comment by alex.bra...@gmail.com on 18 Jun 2013 at 5:33

• Changed state: Invalid

[GoogleCodeExporter]

Of course the best would be to always use UTF-8, but I have an existing 
database to deal with. So it seems to be an issue with Firebird and the default 
charset 'NONE'. For now I stick with my workaround, in the long run I try to 
convert my database to UTF-8. Thanks for your efforts!

Michael

Original comment by michael....@googlemail.com on 18 Jun 2013 at 9:23

[GoogleCodeExporter]

Good luck. Will help if I can.

Alex

Original comment by alex.bra...@gmail.com on 18 Jun 2013 at 11:32

[GoogleCodeExporter]

Thanks!

Original comment by michael....@googlemail.com on 18 Jun 2013 at 5:01