Closed MislavSag closed 1 year ago
That's indeed unfortunate. Does the problem still exist? We use the data.table::between function and, otherwise, data.table functions perform really well. There is nothing special about this function so your workaround with nanotime is fine.
I can't replicate this issue with ~1.2 million rows. These take only 260ms on my computer. Hence, I propose to close this issue. @kboudt Can you close it?
I will try in few days again with some new data I have, but think this can be closed for now. I will reopen if will have similar problem
Hi,
I have downloaded quotes data for SPY from Dukascopy for the test.
The first steps according to SSRN paper include
noZeroQuotes
andexchangeHoursOnly
functions.While
noZeroQuotes
is fast enough,exchangeHoursOnly
is pretty slow. I have 119.304.432 rows. This is quotes data for SPY from 2017. It takes more than 15 minutes (I terminated the operation after cca 15 min).I tried than simple filtering of exchange times using
data.table
andnanotime
packages:where quotes have the same structure as sampleQDataRaw, with only difference that DT is nanotime object and not Posxcit object.
This takes cca 2.5 seconds.
My question is, can I use this simple function or your
exchangeHoursOnly
function do additional filtering ?Important note is that nanotime requires UTC time.