Historical Market Data API - Missing Ticks

[santiagosilvestrini]

Bug description:

Historical Market Data pulled from the API via rRofex package is not matching what is published on the official ROFEX's Statistics site. Not sure if the issue is on the API or on the package itself as I haven't tried to call the API directly.

To Reproduce

While the following reproducible example is for a specific date and symbol, the issue is not isolated only to these parameters.

R Code to get the data from the API

1. Use rRofex::trading_login() function to login with your credentials into the production environment.

   rRofex::trading_login(username = "yourusername", 
   password = "yourpassword", 
   env = "production")

2. Get Historical Market Data for DoJul19 on July 2nd, 2019

datah <- rRofex::trading_mdh(symbol = "DOJul19", date = "2019-07-02")`
nrow(datah) # you should get 598 records

# Perform time conversion from GMT to AR timezone:
datah$datetime <- as.POSIXct(datah$datetime, tz = "GMT")
datah$datetime <- lubridate::with_tz(datah$datetime, tz = "America/Argentina/Buenos_Aires")
datah$date <- as.Date(datah$datetime)

# For easy inspection arrange ticks in decreasing order
datah <- datah %>%
            dplyr::arrange(desc(datetime))

# Take a look to the last 5 ticks
datah[1:5, c("price", "size", "datetime")] %>%
    knitr::kable() %>% 
    kableExtra::kable_styling()

price	size	datetime
43.835	10	2019-07-02 14:59:58
43.820	5	2019-07-02 14:59:52
43.830	10	2019-07-02 14:59:47
43.850	1	2019-07-02 14:59:45
43.870	15	2019-07-02 14:59:45

Get data from Rofex's CEM

3. Open a browser and go to the ROFEX Market Statistics Centre (CEM).

4. Find the Tick by Tick section on the upper menu or use this direct link

5. Use the following search parameters:

• Fecha (date): 02/07/2019. Note it requires DMA format.
• On the second dropdown (Todas los Posiciones), select DLR072019 which is the Clearing symbol for DoJul2019.
• Uncheck Opciones and Combinadas leaving only Futuros checked
• Do not specify any Time range
• Click on Ver

6. You should get a table like the one below:

Posición	Precio	Volumen	Hora
DLR072019	43,8350	10	14:59:58
DLR072019	43,8400	1000	14:59:55
DLR072019	43,8000	1000	14:59:54
DLR072019	43,8000	1000	14:59:54
DLR072019	43,8200	5	14:59:52
...	...	...	...

Comparing Results

• As you will quickly notice, the first tick at 14:59:58 matchs in both size and price.
• But the next 3 records (previous ticks) are quite different.
• While on the CEM we can see 3 ticks of size 1000 between 14:59:54 and 14:59:55, the API is omitting those entries.
• After those 3 ticks we get a match on tick size and price at 14:59:52

• If you continue browsing the list you will find more missing ticks. As a sample, the following are the missing ticks during just the last 10 minutes of operation for same date and symbol:

  datetime	size
  14:58:20	1000
  14:57:49	2000
  14:57:16	2000
  14:56:23	1000
  14:55:30	1000
  14:55:03	1000
  14:54:47	1000
  14:54:32	1000
  14:54:14	1000
  14:51:34	1000
  14:51:33	1000
  14:51:26	1000
  14:51:24	1000
  14:51:24	1000
  ...	...

[santiagosilvestrini]

After additional research I'm closing this issue. What I've found out is that those missing ticks are not actually from DoJul17 but from the wholesale symbol DoJul17A. There is no way to discriminate those values on the CEM but you can easily get them from the API as follows:

datah <- rRofex::trading_mdh(symbol = "DOJul19A", date = "2019-07-02")

As a sample of this new results I got:

price	size	datetime
43.84	1000	2019-07-02 14:59:55
43.80	1000	2019-07-02 14:59:54
43.80	1000	2019-07-02 14:59:54
43.87	1000	2019-07-02 14:58:20
43.87	2000	2019-07-02 14:57:49
...	...	...

Which are exactly the same ticks I was missing at the beginning.

Cheers!

Santi