Closed wwarby closed 5 years ago
No idea why this is happening but in the meantime I've worked around it with the following very ugly hack:
myFunc{[data]
data:update timeColumn:`timestamp$`datetime$timeColumn from data;
...
}
I can afford to do this because I don't need time precision less than 1 second in this particular scenario. Still, would like to know why this is happening so I don't have to use such an ugly hack
Hi @wwarby can you provide some additional information on the node, node-q and q version you use. As well as your operating system.
I see the output as expected:
2019.05.10D00:00:00.000000000
node-q: latest node: 8.x q: KDB+ 3.3 2016.03.14 Copyright (C) 1993-2016 Kx Systems macOS 10.14.4 (18E226)
As you can see, I have stopped using q / kdb+ since quiet a few years and never updated to an up-to-date version :)
Regarding the time zone issue: set the timezone of your node process to UTC. q does not know about time zones. On Linux/macOS: TZ=UTC node ...
Hi @michaelwittig,
Very good of you to keep answering questions on this repo considering you're no longer using q / kdb+ :)
So this does seem to be specific to some dates and not others - as far as I can tell, times that are precisely on the hour.
var nodeq = require('node-q');
nodeq.connect({host: 'localhost', port: 20010}, function(err, con) {
let timestamp = new Date('2019-04-30T23:00:00.000Z');
con.k('{string x}', nodeq.timestamp(timestamp), (err, result) => {
console.log(result); // 2019.04.30D22:59:59.999999872
});
con.k('{string `timestamp$`datetime$x}', nodeq.timestamp(timestamp), (err, result) => {
console.log(result); // Corrects the problem, but really hacky
});
});
So the above should make it reproducible hopefully. I'm using Node v11.11, node-q latest, KDB+ latest (3.6), running Node on Windows 10.
Interestingly I've also noticed that it's always off by precisely 128 nanoseconds - no more, no less.
Could you, in lib/c.js
line 707, replace
wlong(Long.fromNumber(86400000000000 * (x.getTime() / 86400000 - 10957)));
with
wlong(Long.fromNumber(Long.fromString("1000000", false, 10).multiply(Long.fromNumber(x.getTime())).subtract(Long.fromString("946684800000000000", false, 10))));
Hi Michael, sorry it took me a couple of days to get back to you on this but yes, I can confirm that the patch your provided above solves the problem - thanks :)
So It's a bug in c.js then? I'll notify Kx but presumably you can fix it in the node-q library as c.js is an embedded library and not an npm dependency?
c.js is heavily modified. I will provide a new release shortly
released as node-q@2.4.4
Thanks for your bug report!
Brilliant, thanks Michael
Somehow my dates are being corrupted by tiny margins when they are sent to a KDB process. In the very simplest form I can demonstrate the problem with this code:
Output in KDB:
The date is also off by an hour but I haven't tried dealing with the timezone issue yet - for now I need to deal with the weirdness in the nanoseconds. Any ideas why this is happening?