search

KxSystems / pykx

PyKX is a Python first interface to the worlds fastest time-series database kdb+ and it's underlying vector programming language q.
https://code.kx.com/pykx
Other
49 stars 12 forks source link

Passing a DataFrame with datetime64[us] to kdb errors #22

Closed erichards97 closed 6 months ago

erichards97 commented 7 months ago

Describe the bug Calling a KDB function over IPC and providing a DataFrame as an argument that contains a column of datetime64[us] results in an error: TypeError: ktype cannot be inferred from Numpy dtype datetime64[us]

I imagine datetime64[ns] is the standard pandas/numpy type, however we receive us when pulling data from a SQL database using turbodbc (which we're then trying to send to KDB), and converting between the two types feels unnecessary.

To Reproduce q('myfunction', df) where df has a column of datetime64[us]

Expected behavior The df to be converted to a KDB table, with column of type timestamp.

Screenshots N/A

Desktop (please complete the following information):

Additional context N/A

Thanks

erichards97 commented 7 months ago

As a temporary workaround - df['COLUMN'] = df['COLUMN'].apply(pd.Timestamp) before passing to KDB works

rianoc-kx commented 7 months ago

We'll open a ticket internally to add support for datetime[us]

cmccarthy1 commented 6 months ago

Hi @erichards97,

Yesterday we released a fix for this with our 2.4.0 version of PyKX, change to support this is here.

For full release notes see here

erichards97 commented 6 months ago

Great, thank you.