DavZim / RITCH

An R interface to the ITCH Protocol
https://davzim.github.io/RITCH/
Other
18 stars 5 forks source link

Cannot read 12302019.NASDAQ.ITCH #30

Closed DavZim closed 3 weeks ago

DavZim commented 3 weeks ago

returns 0 messages at 8.5 GB file size

DavZim commented 3 weeks ago
file <- "C:/Users/david/Desktop/RITCH-debugger/12302019.NASDAQ_ITCH50"

RITCH::count_messages(file, quiet = FALSE)
#> [Counting]   0 total messages found
#> [Converting] to data.table
#> [Done]       in 0.00 secs at 21.05TB/s
#>     msg_type count
#>       <char> <i64>
#>  1:        S     0
#>  2:        R     0
#>  3:        H     0
#>  4:        Y     0
#>  5:        L     0
#>  6:        V     0
#>  7:        W     0
#>  8:        K     0
#>  9:        J     0
#> 10:        h     0
#> 11:        A     0
#> 12:        F     0
#> 13:        E     0
#> 14:        C     0
#> 15:        X     0
#> 16:        D     0
#> 17:        U     0
#> 18:        P     0
#> 19:        Q     0
#> 20:        B     0
#> 21:        I     0
#> 22:        N     0
#>     msg_type count

system(sprintf("od --format=x1 --read-bytes=10 '%s'", file),
       intern = TRUE)
#> 0000000 00 0c 53 00 00 00 00 0a 11 ea
#> 0000012

# starts with 00 0c (which are ignored), the 53 (hex to char => S message = )
# so we should expect at least one S...

Rcpp::sourceCpp("debug/debug_tools.cpp")

dbg_itch_file(file)
#> Debugging File 'C:/Users/david/Desktop/RITCH-debugger/12302019.NASDAQ_ITCH50' (.gz-file? no)
#> Usage:
#>   - Empty: next message
#>   - Number: for next N messages
#>   - Character: if valid message type, print the next message, e.g., 'A' for add order
#>   - non valid Character: exits the debugging tool
#> Note: Bytes in parenthesis show the first two bytes, which are not used!
#>   Number of Messages:
#>   - 'S': 8906
#> - 'R': 130158
#> - 'H': 3
#> - 'Y': 1701012224
#> - 'L': 924135
#> - 'V': 0
#> - 'W': 0
#> - 'K': 13070618
#> - 'J': 0
#> - 'h': 0
#> - 'A': 3
#> - 'F': 1
#> - 'E': 215089
#> - 'C': 8905
#> - 'X': 15
#> - 'D': 8929
#> - 'U': 1073321
#> - 'P': 11909160
#> - 'Q': 2916816
#> - 'B': 8906
#> - 'I': 0
#> - 'N': 9945
#> =============================
#>   'S' (len 2 + 12) idx    0 at offset     0 (0x0000) | (00 0c) 53 00 00 00 00 0a 11 ea 0e 8c 43 4f 
DavZim commented 3 weeks ago

will close for now, feel free to reopen when needed