No bar found for this time range warning during backtesting

jcity commented 1 year ago

Description

Getting this warning from the backtesting engine using the bar_event callback.

settings.json

{
    "settings": {
        "use_sandbox_websockets": false,
        "websocket_buffer_size": 10000,
        "test_connectivity_on_auth": true,
        "auto_truncate": false,
        "global_shorting": false,
        "coinbase_pro": {
            "cash": "USD"
        },
        "binance": {
            "cash": "USDT",
            "binance_tld": "com"
        },
        "binance_futures": {
            "cash": "USDT",
            "margin_type": "USDT-M"
        },
        "alpaca": {
            "websocket_stream": "iex",
            "cash": "USD",
            "enable_shorting": true,
            "use_yfinance": false
        },
        "oanda": {
            "cash": "USD"
        },
        "okx": {
            "cash": "USDT"
        },
        "keyless": {
            "cash": "USD"
        },
        "kucoin": {
            "cash": "USDT"
        },
        "ftx": {
            "cash": "USD",
            "ftx_tld": "com"
        },
        "ftx_futures": {
            "cash": "USD",
            "ftx_tld": "com"
        },
        "paper": {
            "price_source": "api"
        }
    }
}

backtest.json (if applicable)

{
    "price_data": {
        "assets": []
    },
    "settings": {
        "use_price": "close",
        "smooth_prices": false,
        "GUI_output": false,
        "show_tickers_with_zero_delta": false,
        "save_initial_account_value": true,
        "show_progress_during_backtest": false,
        "cache_location": "/Users/j/.blankly_cache",
        "continuous_caching": true,
        "resample_account_value_for_metrics": "1d",
        "quote_account_value_in": "USD",
        "ignore_user_exceptions": true,
        "risk_free_return_rate": 0.0,
        "benchmark_symbol": null
    }
}

Note: I edited the cache_location here

Error (if applicable)

/usr/local/lib/python3.9/site-packages/blankly/frameworks/strategy/strategy.py:90: UserWarning: No bar found for this time range
  warnings.warn("No bar found for this time range")

Platform Info

Python version: 3.9.13
Platform: 1.18.19b0

Additional context

Repro code:

import pandas as pd

from blankly import Alpaca, Strategy, StrategyState
from datetime import datetime

def init(symbol, state: StrategyState):
    state.variables.history = state.interface.history(
        symbol=symbol, to=150, resolution=state.resolution, return_as="deque"
    )["close"]

def bar_event(bar, symbol, state: StrategyState):
    ts = datetime.utcfromtimestamp(state.time).strftime('%Y-%m-%d %H:%M:%S')
    bar_ts = datetime.utcfromtimestamp(
        bar['time']).strftime('%Y-%m-%d %H:%M:%S')
    print(f"Date: {ts} ({bar_ts}) \t Price: {bar['close']}")

s = Strategy(Alpaca("example-portfolio"))
s.add_bar_event(bar_event, 'SPXL', '1h', init=init)

results = s.backtest(None, {"USD": 10000}, pd.Timestamp(
    '2022-09-29 00:30:00').timestamp(), pd.Timestamp.now().timestamp())

Output:

INFO: "simulate_margin" not specified in preferences, defaulting to: "True"
No cached data found for SPXL from: 1667057400 to 1667139358.668396 at a resolution of 3600 seconds.

Backtesting...
/usr/local/lib/python3.9/site-packages/blankly/frameworks/strategy/strategy.py:90: UserWarning: No bar found for this time range
  warnings.warn("No bar found for this time range")
Date: 2022-09-29 09:00:00 (2022-09-29 08:00:00)          Price: 57.83
Date: 2022-09-29 10:00:00 (2022-09-29 09:00:00)          Price: 58.59
Date: 2022-09-29 11:00:00 (2022-09-29 10:00:00)          Price: 58.44
Date: 2022-09-29 12:00:00 (2022-09-29 11:00:00)          Price: 58.43
Date: 2022-09-29 13:00:00 (2022-09-29 12:00:00)          Price: 57.63
Date: 2022-09-29 14:00:00 (2022-09-29 13:00:00)          Price: 58.2056
Date: 2022-09-29 15:00:00 (2022-09-29 14:00:00)          Price: 56.08
Date: 2022-09-29 16:00:00 (2022-09-29 15:00:00)          Price: 56.82
Date: 2022-09-29 17:00:00 (2022-09-29 16:00:00)          Price: 55.94
Date: 2022-09-29 18:00:00 (2022-09-29 17:00:00)          Price: 55.08
Date: 2022-09-29 19:00:00 (2022-09-29 18:00:00)          Price: 54.8
Date: 2022-09-29 20:00:00 (2022-09-29 19:00:00)          Price: 56.18
Date: 2022-09-29 21:00:00 (2022-09-29 20:00:00)          Price: 56.3615
Date: 2022-09-29 22:00:00 (2022-09-29 21:00:00)          Price: 56.25
Date: 2022-09-29 23:00:00 (2022-09-29 22:00:00)          Price: 56.59
Date: 2022-09-30 00:00:00 (2022-09-29 23:00:00)          Price: 56.84
Date: 2022-09-30 01:00:00 (2022-09-29 23:00:00)          Price: 56.84
/usr/local/lib/python3.9/site-packages/blankly/frameworks/strategy/strategy.py:90: UserWarning: No bar found for this time range
  warnings.warn("No bar found for this time range")
Date: 2022-09-30 09:00:00 (2022-09-30 08:00:00)          Price: 57.98
Date: 2022-09-30 10:00:00 (2022-09-30 09:00:00)          Price: 57.35
Date: 2022-09-30 11:00:00 (2022-09-30 10:00:00)          Price: 56.96
Date: 2022-09-30 12:00:00 (2022-09-30 11:00:00)          Price: 56.57
Date: 2022-09-30 13:00:00 (2022-09-30 12:00:00)          Price: 56.54
Date: 2022-09-30 14:00:00 (2022-09-30 13:00:00)          Price: 56.31
Date: 2022-09-30 15:00:00 (2022-09-30 14:00:00)          Price: 56.79
Date: 2022-09-30 16:00:00 (2022-09-30 15:00:00)          Price: 56.24
Date: 2022-09-30 17:00:00 (2022-09-30 16:00:00)          Price: 56.1
Date: 2022-09-30 18:00:00 (2022-09-30 17:00:00)          Price: 55.325
Date: 2022-09-30 19:00:00 (2022-09-30 18:00:00)          Price: 54.55
Date: 2022-09-30 20:00:00 (2022-09-30 19:00:00)          Price: 53.55
Date: 2022-09-30 21:00:00 (2022-09-30 20:00:00)          Price: 53.89
Date: 2022-09-30 22:00:00 (2022-09-30 21:00:00)          Price: 53.94
Date: 2022-09-30 23:00:00 (2022-09-30 22:00:00)          Price: 53.86
Date: 2022-10-01 00:00:00 (2022-09-30 23:00:00)          Price: 53.86
Date: 2022-10-01 01:00:00 (2022-09-30 23:00:00)          Price: 53.86
/usr/local/lib/python3.9/site-packages/blankly/frameworks/strategy/strategy.py:90: UserWarning: No bar found for this time range
  warnings.warn("No bar found for this time range")
Date: 2022-10-03 09:00:00 (2022-10-03 08:00:00)          Price: 53.63
Date: 2022-10-03 10:00:00 (2022-10-03 09:00:00)          Price: 53.85
Date: 2022-10-03 11:00:00 (2022-10-03 10:00:00)          Price: 54.38
....

Note the (what looks like) a duplicate bar on the last hour of the day

jcity commented 1 year ago

Related questions that I haven't been able to answer from searching the issues and documentation:

Is there a way to get bar events only while the market is open? Ideally, this takes into account holidays and half days.
Why is state.time (the first datetime printed) different from bar['time']?
This is more of a suggestion, but it would be use (at least for debugging) to have the bar open time and close time in the bar parameter

Let me know if I should open a different issues for these questions.

Thanks!

EmersonDove commented 1 year ago

These are good questions I appreciate the careful testing.

I think that most of what you're seeing is how alpaca returns data back to the user and just how we're forced to handle sparse bars. Your original question is about the duplicate bar, in this case you can see its complaining that no data was found. The engine sends a bar event callback anyway with a user warning like you're seeing, so it's recognizing that it has no data for that range and just sends the most recent data. It might look weird because the warning is after the strategy print but this is fairly common when rapidly mixing stderr and stdout outputs.

If you chart bar data you can see the sparsity issue in pre-market and after-market hours:

The only offered guarantee of the state.time variable with bar events is that state.time will be at most one resolution size ahead of the bar time which is the case here.

The bar['time'] attribute is the bar open time, the bar close time is bar['time'] + resolution.

There might be some subtle timing bugs lurking but with just a quick look over it seems to be expected behavior.

We use this bit of code to filter on our end when doing stock bar events:

    timestamp = datetime.datetime.fromtimestamp(bar['time']).astimezone(tz=timezone.utc)
    if timestamp.weekday() >= 5 or timestamp.time() < START or timestamp.time() > END:
        return

where

START = datetime.time(13, 30)
END = datetime.time(20, 00)

blankly-finance / blankly