ClickHouse / clickhouse-odbc

ODBC driver for ClickHouse
https://clickhouse.tech
Apache License 2.0
245 stars 86 forks source link

Infinite memory consumption, clickhouse-odbc + pyodbc #348

Open xm-repo opened 3 years ago

xm-repo commented 3 years ago

Environment

Issue

Hello, thx for great library! I think there's something wrong going on with memory reallocation in ReadVarColumn, consider test like this:

import pyodbc

def query(q):

    print("{} :".format(q))
    cursor = pyodbc.connect('DSN=ClickhouseTest').cursor().execute(q)

    for i, row in enumerate(cursor):
        print(i, row)

if __name__ == '__main__':
    query("SELECT repeat('a', 5000)")

where DSN looks like this

[ClickhouseTest]
Driver = ClickHouse ODBC Driver
Server = play-api.clickhouse.tech
Database = datasets
UID = playground
PWD = clickhouse
Port = 8443
Proto = https
Timeout=60
SSLMode = allow

In my system this script hangs out and eats all available memory.

And this is my trace, which also repeats infinitely

[ODBC][12501][1612779462.632364][SQLGetData.c][237]
        Entry:
            Statement = 0x5570d7dc4490
            Column Number = 1
            Target Type = -8 SQL_WCHAR
            Buffer Length = 4102
            Target Value = 0x7f257b78f4ec
            StrLen Or Ind = 0x7fff37fb3370
[ODBC][12501][1612779462.632408][SQLGetData.c][545]
        Exit:[SQL_SUCCESS_WITH_INFO]                
            Buffer = [aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa](unicode)                
            Strlen Or Ind = 0x7fff37fb3370 -> 8194