search

jshinonome / vscode-q

vscode kdb+/q extension
GNU General Public License v3.0
41 stars 16 forks source link

Extension incorrectly parses int64 type #97

Open Yanpas opened 5 months ago

Yanpas commented 5 months ago

We store in one column int64 values.

In grid view last two digits become zeroes. In visualization they are rendered as int32_min.

E.g. 1608532083302613811 in grid view becomes 1608532083302613800. In visualization: -2,147,483,648

jshinonome commented 5 months ago

It is because javascript doesn't support int64 type. You have to convert it to string before load them in grid or virtualization mode.

Yanpas commented 5 months ago

The bad thing is that extension silently misleads about the data that's contained in the table (this big number in the example was an id).

JS has BigInt class to deal with larger integer types.

Also I'm fine with treating it as string but I guess sorting will work incorrect then.

jshinonome commented 5 months ago

The underlying lib node-q doesn't use bigint. Though my lib (jkdb) uses bigint, sending json message to grid or virtualization still has to convert bigint to general number. I will think if it is possible to send arrow format. However, I am less likely to implement the feature for vscode-q. You could try kx's official plugin and ask them to fix it.

Yanpas commented 5 months ago

This is already available in vscode-q from a quick overlook: https://github.com/michaelwittig/node-q?tab=readme-ov-file#types-footnote5

There's a long2number parameter of q.ConnectionParameters

jshinonome commented 5 months ago

Grid/virtualization mode is an independent html page from vscode-q conn manager, when vscode-q got data from kdb, it has to send data to grid/virtualization html page. The data conversion caused losing precision. Sending as string data type to grid/virtualization page, the chart cannot display properly.

Yanpas commented 5 months ago

Side question: do debug breakpoints for typescript code of the extension (client) work in your environment? Looks like mine vscode can't find sourcemaps: image The diagnostic tells:

We couldn't find a corresponding source location, and didn't find any source with the name client.ts.
How did you expect this file to be loaded? (If you have a compilation step, you should pick 'sourcemap')

I've tried both Webpack & Launch Extension and npx tsc and then Launch Extension

PS want to experiment with https://github.com/sidorares/json-bigint or encoding everything as an object with manual deserialization on webview side