Failed to save modifications to the server. Error returned: 'OLE DB or ODBC error: [DataSource.Error] ERROR [HY000] Incomplete input stream, expected at least 28 more bytes.

[AlexanderFokinImpr]

Hi! I've got an error when I try to get data to Power BI from Clickhouse through ODBC connector. It can be seen that the data starts to load, but after a few seconds this error appears: Failed to save modifications to the server. Error returned: 'OLE DB or ODBC error: [DataSource.Error] ERROR [HY000] Incomplete input stream, expected at least 28 more bytes.

Timeout ODBC = 900000

Query is really simple, like: SELECT * FROM scheme1.table1 LIMIT 1000000

scheme1.table1 has about 15 mln of rows.

Could you suggest what is the reason of that issue?

[kobzevvv]

Have the same problem!

[d4rkr4in]

So annoying, the same issue. It can sometimes happen sometimes not.

[d4rkr4in]

Hey guys, try to NOT use SQL Query in Power Query. Use native buttons for that. Get Source - Additional - ODBC - OK (don't write any SQL query). It should help you. Try and get feedback

[GrabsOrion]

Same problem, but with Informatica PowerCenter.

[nikerov-kirill]

Same problem. Error OLE DB or ODBC: [DataSource.Error] ERROR [HY000] Incomplete input stream, expected at least 201 more bytes. Does anyone have a solution?

[akzhar]

Same problem. Error: PipelineException: ERROR [HY000] Incomplete input stream expected at least 4 more bytes Have no idea what's the reason... It sometimes happen sometimes not.

[vladdell]

Same problem with SSAS Tabular.

[pauliusmiskunas]

Similar problem here, trying to get data to Qlik Sense from Clickhouse through ODBC connector.

The following error occurred:
QVX_UNEXPECTED_END_OF_DATA: SQL##f - SqlState: HY000, ErrorCode: 1, ErrorMsg: Incomplete 
input stream, expected at least 1192 more bytes

It can sometimes happen sometimes not. When load is finished without error, the number of rows doesn't match the table rows number in database (not all rows are loaded), also getting different number of records each load. ODBC version - Release 1.2.1.20220905 ClickHouse - 24.3.2.23 table DDL:

CREATE TABLE db_name.table_name
(
    `id` UInt64,
    `title` Nullable(String),
    `item_price` Nullable(Float64),
    `description` Nullable(String),
    `image_url` Nullable(String),
    `product_url` Nullable(String),
    `manufacturer_code` Nullable(String),
    `article_number` Nullable(String),
    `sale_price` Nullable(Float64),
    `manufacturer` Nullable(String),
    `short_message` Nullable(String),
    `specs` Nullable(String),
    `brand` Nullable(String),
    `categories` Nullable(String),    
    `margin_priority` Nullable(String),
    `eans_values` Nullable(String),
    `updated_at` DateTime
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(updated_at)
ORDER BY id
SETTINGS index_granularity = 8192;

[paulorijnberg]

I am getting the same error. For some context:

• I got a fact model with ~ 26M records.
• The error only occurs when I add an additional column via Power Query. The column is simply multiplying the units with the price.
• I am currenly using the import mode, since we are stumbling upon a Clickhouse 500 ERROR when creating relationships between fact tables and bridge tables (we are following Kimballs architecture).

When adding the given column I get the following error:

Found a workaround; I noticed when adding the column via DAX, the error doesn't appear. Note, I think adding a column via DAX might sometimes be less efficient. Also, I don't think this column is included in the data model but only created in the report. So if you are creating a single data model with multiple dashboards, you might need to add the DAX column multiple times.

Hopefully this will be fixed soon!

[Slach]

try to execute in clickhouse-client SELECT ... FORMAT Null maybe issue is not related to ODBC and related to your selected tables data

[mshustov]

To start the investigation, we need more information to reproduce the problem:

1. ClickHouse and driver versions
2. OS
3. query log for the period when the problem happens
4. the driver logs for the period when the problem happens. The logging might be enabled via DriverLog and DriverLogFile DSN parameters: https://github.com/ClickHouse/clickhouse-odbc/blob/2f55d016420af9dfbd503ba122c0c1549c180363/packaging/odbc.ini.sample#L36-L37
5. DDL for a query target table

[MindaugasDatanome]

@mshustov , could you please specify how to enable logging in Windows Server 2016 STD ? The settings in the screenshot don't seem to log anything.

It would be great if you could resolve this issue, as it's an essential functionality of the driver. I can assist with gathering more detailed information and reproducing scenarios where data is not fully captured, as we can easily replicate such situations.

[mshustov]

could you please specify how to enable logging in Windows Server 2016 STD ? The settings in the screenshot don't seem to log anything.

@slvrtrn, could you help with enabling logging?

[slvrtrn]

Can you please try adding the following to the connection string? This should enable logging at the specified path

[MindaugasDatanome]

The log file with the error [HY000 (Incomplete input stream, expected at least 544503151 more bytes)] is attached. There was no error message on the BI client side, but only part of the data was read, not the full table.

DDL of this table:
CREATE TABLE table_name
(
    `id` UInt64,
    `title` Nullable(String),
    `item_price` Nullable(Float64),
    `description` Nullable(String),
    `image_url` Nullable(String),
    `product_url` Nullable(String),
    `manufacturer_code` Nullable(String),
    `article_number` Nullable(String),
    `sale_price` Nullable(Float64),
    `manufacturer` Nullable(String),
    `short_message` Nullable(String),
    `specs` Nullable(String),
    `brand` Nullable(String),
    `categories` Nullable(String),
    `margin_priority` Nullable(String),
    `eans_values` Nullable(String),
    `updated_at` DateTime,
    `uuid` UUID DEFAULT generateUUIDv4()
)
ENGINE = MergeTree
ORDER BY id
SETTINGS index_granularity = 8192;

clickhouse-odbc_incomplete input stream.zip

[mshustov]

@MindaugasDatanome, Thank you for reporting your DDL! Would it be possible to provide a small dataset to help us reproduce the problem? You can use https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp to hide sensible data. Unfortunately, the logs aren't that helpful for troubleshooting 😞

[MindaugasDatanome]

I'm attaching a snippet of data from this table. What's also interesting is that each time when read the same table, a different number of records is read before it stops.

3 attempts to read: 842700 out of 31647840; 831500 out of 31647840; 843100 out of 31647840. data_for_investigation.zip

[glebbuk]

@paulorijnberg thanks! In my case this error did not appear again when I deleted additional column in PowerQuery. @mshustov could you please look at the proccess of adding new columns not in sql but PowerQuery driver?

[mshustov]

@mshustov could you please look at the proccess of adding new columns not in sql but PowerQuery driver?

@glebbuk could you elaborate?