Open ajennings67 opened 5 years ago
Can you please provide some minimal example of python code using some standard libs, so we can check/reproduce that? It it not very handy to reconstruct the code from screenshots and guess what those function df_to_ch, query_df etc. do and where do they came from.
You can just upload some snippet, and fragments of H:\BOND_TRA\ATJ\Quant\Projects\DMA\clickhouse.py
to gist. Thank you.
Meanwhile, I tried with ODBCTest
tool (in Windows) and isql
tool (in Linux), and both, null
's and nan
's seem to work with either of those tools as expected in the recent code. I was trying with changes from this PR https://github.com/ClickHouse/clickhouse-odbc/pull/211 applied to the master
branch locally, but the nature of the involved code suggests that the latest release (v1.1.1.20191108
) should work in this case too.
Table created, populated, and verified as:
$ clickhouse-client --query "CREATE TABLE nullable_nan_col_issue_tbl ( nullable_nan_col Nullable(Float64) ) ENGINE = Memory"
$ clickhouse-client --query "INSERT INTO nullable_nan_col_issue_tbl VALUES ( NULL ), ( NAN ), (1.23456789)"
$ clickhouse-client --query "SELECT * FROM nullable_nan_col_issue_tbl"
\N
nan
1.2345678900000001
ODBCTool
data retrieval result:
Full Connect(Default)
Env. Attr. SQL_ATTR_ODBC_VERSION set to SQL_OV_ODBC3
Successfully connected to DSN 'ClickHouse_DSNW_Debug'.
SQLExecDirect:
In: Statementhandle = 0x0000000002064290, StatementText = "SELECT * FROM nullable_nan_col_issue_tbl", Statementlength = 40
Return: SQL_SUCCESS=0
Get Data All:
"nullable_nan_col"
<Null>
nan
1.2345678900000001
3 rows fetched from 1 column.
isql
data retrieval result:
$ isql ClickHouse_DSNW_Debug
+---------------------------------------+
| Connected! |
| |
| sql-statement |
| help [tablename] |
| quit |
| |
+---------------------------------------+
SQL> SELECT * FROM nullable_nan_col_issue_tbl
+-------------------+
| nullable_nan_col |
+-------------------+
| |
| nan |
| 1.2345678900000001|
+-------------------+
SQLRowCount returns 3
3 rows fetched
SQL>
I have been experiencing some issues when trying to query data that contains null values via the Clickhouse ODBC Driver for Windows. I have tested reading and writing from both Ubuntu and Windows. Reading from Windows always fails and reading from Ubuntu always succeeds, regardless of which platform was used to write.
For the sake of space, I won't include the utility I wrote to read and write to CH, but I can provide details as needed. It uses the following format to write:
and the following format to read:
where the connection uses the Clickhouse ODBC Driver for Windows.
I use the datatype Nullable(Float64) for all the columns in my table.
Expected Behavior
Create example, write, and read via Ubuntu:
Actual Behavior
Using the same example from above, except we are using the Windows ODBC Driver:
Error:
To ensure this isn't an issue related to writing via Window, here we use windows to read the table written by Linux:
Further, linux can read the data written by windows:
Now that is issue is specific to Windows reading, we can see that the issue stems from the Nan...
No Nan in query results:
Nan included in query results
Summary
It appears that the Windows ODBC Driver is not able to handle Nan values in query result
Thanks in advance and please let me know if I can provide additional information.