search

man-group / arctic

High performance datastore for time series and tick data
https://arctic.readthedocs.io/en/latest/
GNU Lesser General Public License v2.1
3.06k stars 583 forks source link

out-of-order in tickStore #732

Closed DennyZen closed 5 years ago

DennyZen commented 5 years ago

i am using Arctic ver 1.73.0 TickStore on Windows Server 2016

saving orderbook ticks stream to db.. quite big flow(20-50 ticks/sec).. before it was good. but i add threading.Timer function that produce one additional tick every 1 second I got "UnorderedDataException timestamps out-of-order " after some hours now. i use array for buffering ticks. during adding this array to db - error occurs 

> msg['index'] = datetime.now()
> self.data.append(msg)
> if len(self.data)==100000:
>     self.lib.write(self.sym, self.data)

my theory is: when we got new tick, formed new message, added time to it, but still not append to array - if in this moment another process(Timer) runs - maybe we'll have this kind of error.. (delay in out of order tick is very small - ~10ms) What you think? Any suggestions?

DennyZen commented 5 years ago

already fixed it! i put my additional ticks to another collection. Easy

bmoscon commented 5 years ago

so we can close this?