Scheduled Week Start Events Fire on Wrong Day for Futures for Dates Prior to 2023-12-26

jdbates commented 1 week ago

Expected Behavior

When scheduling a week_start event using a future contract with no offset, the event should fire on the first trading day of each week.

Actual Behavior

For backtests beginning prior to 2024, the week_start event for futures fires one day prior to the first trading day. Thus, scheduled events using self.time_rules.after_market_open will not trigger unless you pass offset=1 to self.date_rules.week_start.

Potential Solution

Manually setting offset=1 in self.date_rules.week_start is a workaround, but only if your backtest ends prior to 2023-12-26. Also, this creates an inconsistency with the behavior for equities, which fire on the correct day.

If the backtest includes dates both prior to and post 2023-12-16, the workaround would need to delete the scheduled event with offset=1 and replace it with an event with no offset to ensure that the event is triggering on the correct day both before and after that date.

Reproducing the Problem

class WeekStartTest(QCAlgorithm):

def initialize(self):
    # Set Backtest Properties
    self.set_start_date(2020, 1, 1)
    self.set_end_date(2024, 1, 7)

    self.future = self.add_future(
        Futures.Indices.SP_500_E_MINI, Resolution.MINUTE,
        data_normalization_mode=DataNormalizationMode.BACKWARDS_PANAMA_CANAL,
        data_mapping_mode=DataMappingMode.OPEN_INTEREST,
        contract_depth_offset=0
    )
    self.equity = self.add_equity("SPY")

    self.schedule.on(
        self.date_rules.week_start(self.equity.symbol, 0),
        self.time_rules.after_market_open(self.future.symbol, 0),
        lambda: self.log("Equity Market Open")
    )
    self.schedule.on(
        self.date_rules.week_start(self.future.symbol, 0),
        self.time_rules.after_market_open(self.future.symbol, 0),
        lambda: self.log("Future Market Open - Offset 0")
    )
    self.schedule.on(
        self.date_rules.week_start(self.future.symbol, 1),
        self.time_rules.after_market_open(self.future.symbol, 0),
        lambda: self.log("Future Market Open - Offset 1")
    )

Note that the offset=0 event for futures won't fire until the backtest reaches 2023-12-26, so prior to that date the only logs produced are "Equity Market Open" and "Future Market Open - Offset 1", both of which are recorded on the same day. After 2023-12-26, all three logs get created, with "Equity Market Open" coinciding with "Future Market Open - Offset 0" and "Future Market Open - Offset 1" occurring the next day.

System Information

QC Cloud

Checklist

[x] I have completely filled out this template
[x] I have confirmed that this issue exists on the current master branch
[x] I have confirmed that this is not a duplicate issue by searching issues
[x] I have provided detailed steps to reproduce the issue

jdbates commented 1 week ago

On further research, it looks like the bug persists after 2023-12-26 as well. It just so happens that for the week of 2023-12-26 and 2024-1-2, the event fires on the correct day for both futures and equities. After that, it reverts back to firing a day early for futures.

I'm guessing the reason it fired correctly on those two weeks had to do with the holiday scheduling lining up with week start.

jdbates commented 1 week ago

After a little more digging, it looks like the root cause of this is an inconsistency in the behavior of the week_start date rule and the after_market_open time rule. Since most futures use extended market hours, the week_start method is firing on Sunday (correctly). However, the after_market_open time rule is still using regular market hours, so it never fires (except for rare cases where holidays move the Futures market week_start event to a week day).

I don't know if this is intended behavior for each rule, but if it is there needs to be some clarification about which rules respect extended market hours and which don't. The after_market_open rule not only fails to fire, but it does so silently - not even a warning is given.

QuantConnect / Lean