Postgres client prints error and no data on character set conversion issue

[ptasheq]

Issue report

Hi Everyone,

On querying some foreign tables in Postgres I'm facing the following issue:

=# SELECT * FROM mytable;
NOTICE:  tds_fdw: Query executed correctly
NOTICE:  tds_fdw: Getting results
ERROR:  DB-Library error: DB #: 2403, DB Msg: Some character(s) could not be converted into client's character set.  Unconverted bytes were changed to question marks ('?'), OS #: 0, OS Msg: Success, Level: 4

The error prevents Postgres client from showing any results. Interestingly, when I'm using tsql I also get this message, but records are still fetched:

2> SELECT * FROM mytable;
3> GO
Error 2403 (severity 4):
    Some character(s) could not be converted into client's character set.  Unconverted bytes were changed to question marks ('?')
1955 2 237 701-02 1931.44 450.00 2 płyta EUR 1 4.29208889 1
25000 20366 5647 149 9300.00 0.00 1 wp?ata PLN NULL 0.00000000 1

The majority of records are displayed properly (with accents/diacritics), question marks occur rarely. Actually, that would be acceptable, so my question is: is it possible to suppress this error to get the results in Postgres? Or are there any other solutions for this? I played around with the server settings in freetds.conf by trying other client charsets and tds versions without success.

The server entry in freetds.conf:

[myserver]
    host = <server_ip>
    port = 1433
    tds version = 7.4
    client charset = UTF-8

Operating system

Ubuntu 16.04.7 LTS

Version of tds_fdw

Name        | tds_fdw
Version     | 2.0.2
Schema      | public
Description | Foreign data wrapper for querying a TDS database (Sybase or Microsoft SQL Server)

Version of PostgreSQL

PostgreSQL 9.5.25 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609, 64-bit

Version of FreeTDS

AC_INIT(FreeTDS, 1.3.2)

Logs

Sentences, data structures, data

[robins]

ptasheq@ is it possible to have a reproducible use-case here? Ideally 1-2 lines of INSERT statements for MSSQL that could be tested and reproduced? That allows for someone else to see what you may be seeing here.

[Jasopaum]

Any update on this one?