QuantConnect / Lean

Lean Algorithmic Trading Engine by QuantConnect (Python, C#)
https://lean.io
Apache License 2.0
9.57k stars 3.23k forks source link

Bug in History request for Tick data of Future #7853

Open scloudyy opened 6 months ago

scloudyy commented 6 months ago

Expected Behavior

This 3 History requests for Tick data should return same number of Tick data for Future

  1. History(Symbol, start, end, Resolution.Tick)
  2. History[Tick](Symbol, start, end, Resolution.Tick)
  3. History(Tick, Symbol, start, end, Resolution.Tick)

Actual Behavior

When requesting Tick data for Future, only the first History request History(Symbol, start, end, Resolution.Tick) returns data

The second and third History request returns empty or a few unrelated data point.

Reproducing the Problem

Run below code on research environment in Algorithm Lab:

qb.SetStartDate(2020, 6, 4)
future = qb.AddFuture(Futures.Currencies.EUR, Resolution.Minute)
contract_symbols = sorted(qb.FutureChainProvider.GetFutureContractList(future.Symbol, qb.Time), key=lambda symbol: symbol.ID.Date)
contract = contract_symbols[0]
qb.AddFutureContract(contract_symbols[0])

end_time = datetime(2020, 6, 3, 11, 0, 0)
start_time = end_time - timedelta(days=1)
ticks_df_tick_in_first_argument = qb.History(Tick, contract, start_time, end_time, Resolution.Tick)
ticks_obj = qb.History[Tick](contract, start_time, end_time, Resolution.Tick)
ticks_df = qb.History(contract, start_time, end_time, Resolution.Tick)

print('length of tick dataframe:' + str(len(ticks_df)))
print('length of tick dataframe with Tick in first argument:' + str(len(ticks_df_tick_in_first_argument)))
print('length of tick object list:' + str(len([t for t in ticks_obj])))

// The output is:
// length of tick dataframe:856834
// length of tick dataframe with Tick in first argument:3
// length of tick object list:3

The History request History(Symbol, start, end, Resolution.Tick) returns 856834 data points. The other returns 3 data points, which are all Open Interest instead of Trade or Quote, see image below:

Screenshot 2024-03-14 at 2 24 59 PM

Also tried different future like Futures.Indices.SP500EMini, as well as different time range 2024-03-06 11:00:00 - 2024-03-07 11:00:00. The issue consists for multiple Futures and multiple time range:

Screenshot 2024-03-14 at 2 25 08 PM Screenshot 2024-03-14 at 2 25 12 PM

To make the second and third History request return more data, the future must be added at Tick resolution qb.AddFutureContract(contract_symbols[0], Resolution.Tick). After that 3 History call return same data:

Screenshot 2024-03-17 at 3 35 17 PM

This issue not exist in Equity. Take SPY for example. SPY are added at Minute resolution and 3 History call returns same amount Tick data:

Screenshot 2024-03-14 at 2 25 19 PM

System Information

Algorithm Lab

Checklist

scloudyy commented 6 months ago

Also tested Crypto. Looks like Crypto also don't have this issue. The Crypto is added add Minute solution and 3 History call for Tick returns same data:

Screenshot 2024-03-17 at 3 24 52 PM

Looks like issue only happens to Future.

Martin-Molinero commented 6 months ago

Hey @scloudyy! Thank you for the detail report, we will check it out when we have a chance