Closed MathieuBartels closed 10 months ago
I sadly spent time doing something almost exactly the same as this PR; I guess I should have checked here first :).
I also updated around line 1662 to include 2 new fields, "agg" (Trade Aggressor) and "day" (Day Code); I'm particularly interested in getting Trade Aggressor info:
tick_type = np.dtype([
('tick_id', 'u8'),
('date', 'M8[D]'), ('time', 'm8[us]'),
('last', 'f8'), ('last_sz', 'u8'),
('last_type', 'S1'), ('mkt_ctr', 'u4'),
('tot_vlm', 'u8'), ('bid', 'f8'), ('ask', 'f8'),
('cond1', 'u1'), ('cond2', 'u1'), ('cond3', 'u1'), ('cond4', 'u1'),
('agg', 'u8'), ('day', 'u8') # 6.1
])
...and line 1813:
if len(dl) > 11:
data[line_num]['agg'] = np.uint8(dl[11])
if len(dl) > 12:
data[line_num]['day'] = np.uint8(dl[12])
...the rest were Python formatting-related changes (and starting work for MarketDepthConn).
Capturing ticks iteratively into a DataFrame and dumping to a Pickle file, only to subsequently re-export as SQLite3, I ended up with a SQL database that I can read quickly from Python (or lately Rust):
-- ticks definition
CREATE TABLE "ticks" ( "tick_id" INTEGER, "date" TIMESTAMP, "time" TEXT, "last" REAL, "last_sz" INTEGER, "last_type" TEXT, "mkt_ctr" INTEGER, "tot_vlm" INTEGER, "bid" REAL, "ask" REAL, "cond1" INTEGER, "cond2" INTEGER, "cond3" INTEGER, "cond4" INTEGER, "agg" INTEGER, "day" INTEGER );
Support for protocol version 6.1