akapur / pyiqfeed

Python LIbrary for reading DTN's IQFeed
GNU General Public License v2.0
168 stars 108 forks source link

Support 6.1 #62

Closed MathieuBartels closed 10 months ago

MathieuBartels commented 1 year ago

Support for protocol version 6.1

GuyResh commented 11 months ago

I sadly spent time doing something almost exactly the same as this PR; I guess I should have checked here first :).

I also updated around line 1662 to include 2 new fields, "agg" (Trade Aggressor) and "day" (Day Code); I'm particularly interested in getting Trade Aggressor info:

tick_type = np.dtype([
    ('tick_id', 'u8'),
    ('date', 'M8[D]'), ('time', 'm8[us]'),
    ('last', 'f8'), ('last_sz', 'u8'),
    ('last_type', 'S1'), ('mkt_ctr', 'u4'),
    ('tot_vlm', 'u8'), ('bid', 'f8'), ('ask', 'f8'),
    ('cond1', 'u1'), ('cond2', 'u1'), ('cond3', 'u1'), ('cond4', 'u1'),
    ('agg', 'u8'), ('day', 'u8')  # 6.1
])

...and line 1813:

            if len(dl) > 11:
                data[line_num]['agg'] = np.uint8(dl[11])
            if len(dl) > 12:
                data[line_num]['day'] = np.uint8(dl[12])

...the rest were Python formatting-related changes (and starting work for MarketDepthConn).

Capturing ticks iteratively into a DataFrame and dumping to a Pickle file, only to subsequently re-export as SQLite3, I ended up with a SQL database that I can read quickly from Python (or lately Rust):

-- ticks definition

CREATE TABLE "ticks" ( "tick_id" INTEGER, "date" TIMESTAMP, "time" TEXT, "last" REAL, "last_sz" INTEGER, "last_type" TEXT, "mkt_ctr" INTEGER, "tot_vlm" INTEGER, "bid" REAL, "ask" REAL, "cond1" INTEGER, "cond2" INTEGER, "cond3" INTEGER, "cond4" INTEGER, "agg" INTEGER, "day" INTEGER );