exxeleron / qPython

interprocess communication between Python and kdb+
http://www.devnet.de
Apache License 2.0
152 stars 89 forks source link

Null float to KDB+ converts to empty string #50

Open dberrios2000 opened 7 years ago

dberrios2000 commented 7 years ago

I am currently using python 2.7.12 using qpython 1.2.2, numpy 1.10.4.

Running the following script does not send a NAN float datatype to KDB+ but instead an empty string.

from qpython.qtype import qnull,QDOUBLE,QFLOAT,_QNAN64 from qpython import qconnection import numpy

q = qconnection.QConnection(host = 'localhost', port = 5010) q.open() q.sync('foo',numpy.string_('testtrade'),[numpy.string('PYTHON'),qnull(QDOUBLE)]) or q.sync('foo',numpy.string_('testtrade'),[numpy.string('PYTHON'),_QNAN64])

KDB+ 3.3 2015.11.03 : q)foo:{[x;y] show y} q)IBM " " 2nd example: q.sync('foo',numpy.string_('test_trade'),[numpy.string_('IBM'),qnull(-7)]) q)IBM 0N

Seems like a bug when trying to send a null float datatype .

dberrios2000 commented 7 years ago

Does anyone know why sending q.sync('foo', numpy.string_('testtrade'),[numpy.string('PYTHON'),numpy.fromstring(_QNAN64)]) would send KDB a value of ,0n

maciejlach commented 7 years ago

Hi @dberrios2000

it looks like there is a bug connected with Pandas integration. When serializing generic lists, NaN values are converted to single space.

Before the patch is available and pulled by the maintainer, please use this workaround:

from qpython.qtype import qnull, QDOUBLE
from qpython import qconnection
from qpython.qwriter import QWriter

with qconnection.QConnection(host = 'localhost', port = 5000, writer_class = QWriter) as q:
  q.sync('foo', 'table', ['PYTHON', qnull(QDOUBLE)])

This enforces qpython to use default, non-pandas serializer, even if the pandas package is installed.

Best regards Maciej

dberrios2000 commented 7 years ago

This worked.. thank you for your help.