IBKR Data Downloader Exception

jaredbroad commented 6 months ago

Expected Behavior

Able to download options data on IBKR

Actual Behavior

Runtime exception looking up IBKR Symbols

20240514 21:08:17.198 TRACE:: InteractiveBrokersBrokerage.LookupSymbols(): Requesting symbol list for AAPL ... 20240514 21:08:17.218 ERROR:: <>c__DisplayClass9_0.b__0(): System.NullReferenceException: Object reference not set to an instance of an object. at QuantConnect.Brokerages.InteractiveBrokers.InteractiveBrokersBrokerage.LookupSymbols(Symbol symbol, Boolean includeExpired, String securityCurrency) at QuantConnect.ToolBox.IBDownloader.IBDataDownloader.GetChainSymbols(Symbol symbol, Boolean includeExpired) at QuantConnect.ToolBox.IBDownloader.IBDataDownloader.Get(DataDownloaderGetParameters dataDownloaderGetParameters)

Potential Solution

Review and address

Reproducing the Problem

Excellent post here: https://www.quantconnect.com/forum/discussion/17071/download-and-backtest-options-data-locally-with-lean-cli-and-interactive-brokers/p1

Checklist

[x] I have completely filled out this template
[x] I have confirmed that this issue exists on the current master branch
[x] I have confirmed that this is not a duplicate issue by searching issues
[x] I have provided detailed steps to reproduce the issue

omidkrad commented 6 months ago

You can use this algo to reproduce it.

BreakoutCallBuySample.cs

using System;
using System.Linq;
using QuantConnect;
using QuantConnect.Util;
using QuantConnect.Algorithm;
using QuantConnect.Indicators;
using QuantConnect.Data;
using QuantConnect.Data.Market;
using QuantConnect.Orders;
using QuantConnect.Securities;

// Source: https://www.quantconnect.com/learning/task/257/Options-Trading
public class Main : QCAlgorithm
{
    private Symbol _equity;
    private OptionContract _call;
    private Maximum _high;

    public override void Initialize()
    {
        SetStartDate(2024, 5, 6);
        SetEndDate(2024, 5, 14);

        SetCash(100000);

        _equity = AddEquity("AAPL", Resolution.Minute).Symbol;
        Securities[_equity].SetDataNormalizationMode(DataNormalizationMode.Raw);
        SetBenchmark(_equity);

        var option = AddOption("AAPL", Resolution.Minute);
        option.SetFilter(-3, 3, TimeSpan.FromDays(20), TimeSpan.FromDays(40));

        _high = MAX(_equity, 21, Resolution.Daily, Field.High);
    }

    public override void OnData(Slice data)
    {
        if (!_high.IsReady)
            return;

        var optionInvested = Portfolio.Keys
            .Where(x => Portfolio[x].Invested && Portfolio[x].Type == SecurityType.Option)
            .ToList();

        if (optionInvested.Any())
        {
            if (Time + TimeSpan.FromDays(4) > optionInvested[0].ID.Date)
            {
                Liquidate(optionInvested[0], "Too close to expiration");
            }
            return;
        }

        if (Securities[_equity].Price >= _high.Current.Value)
        {
            foreach (var optionChain in data.OptionChains)
            {
                var chains = optionChain.Value;
                BuyCall(chains);
            }
        }
    }

    private void BuyCall(OptionChain chains)
    {
        var expiry = chains.OrderBy(x => x.Expiry).Last().Expiry;
        var calls = chains.Where(x => x.Expiry == expiry && x.Right == OptionRight.Call);
        var callContracts = calls.OrderBy(x => Math.Abs(x.Strike - x.UnderlyingLastPrice));

        if (!callContracts.Any())
            return;

        _call = callContracts.First();
        var quantity = Portfolio.TotalPortfolioValue / _call.AskPrice;
        quantity = Math.Floor(0.05m * quantity / 100);
        Buy(_call.Symbol, quantity);
    }

    public override void OnOrderEvent(OrderEvent orderEvent)
    {
        var order = Transactions.GetOrderById(orderEvent.OrderId);
        if (order.Type == OrderType.OptionExercise)
        {
            Liquidate();
        }
    }
}

Martin-Molinero commented 6 months ago

Update: at the moment IB does not provide the option symbol chain, so we can't source the symbol list to download, causing the error reported

omidkrad commented 6 months ago

Thank you @Martin-Molinero for the update. I think we should show that in the brokerages page that which brokers support requesting option chain, in addition to support for trading options.

Looking at IBKR docs they have an API to provide option chain: https://ibkrcampus.com/ibkr-api-page/twsapi-doc/#option-chain

omidkrad commented 6 months ago

I noticed there are two calls to the reqContractDetails method in InteractiveBrokersBrokerage.cs I tried to set breakpoints on them, the process attaches but these calls didn't get hit.

omidkrad commented 6 months ago

You can reduce the repro project to the following:

using System;
using QuantConnect;
using QuantConnect.Algorithm;
using QuantConnect.Data;

public class Main : QCAlgorithm
{
    public override void Initialize()
    {
        SetStartDate(2024, 5, 13);
        SetEndDate(2024, 5, 15);
        SetCash(100000);

        var equity = AddEquity("SPY", Resolution.Minute).Symbol;
        Securities[equity].SetDataNormalizationMode(DataNormalizationMode.Raw);
        SetBenchmark(equity);

        var option = AddOption("SPY", Resolution.Minute);
        option.SetFilter(-5, 5, TimeSpan.FromDays(20), TimeSpan.FromDays(40));
    }

    public override void OnData(Slice data)
    {
        foreach (var (symbol, optionChain) in data.OptionChains)
        {
            foreach (var option in optionChain)
            {
                Log($"Received {option}");
            }
        }
    }
}

omidkrad commented 4 months ago

Related issues for IQFeed and Polygon:

omidkrad commented 3 months ago

Since a recent update in August I'm getting a different error when running local backtesting:

lean backtest .\MyOptionAlgo --data-provider-historical "Interactive Brokers"

Error:

Unhandled exception. System.Reflection.TargetInvocationException: Exception has been thrown by the target of an
invocation.
 ---> System.ArgumentException: Unable to locate any exports matching the requested typeName:
QuantConnect.DownloaderDataProvider.Launcher.Models.BrokerageDataDownloader (Parameter 'typeName')
   at QuantConnect.Util.Composer.GetExportedValueByTypeName[T](String typeName, Boolean forceTypeNameOnExisting) in
/LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Common/Util/Composer.cs:line 317
   at QuantConnect.Lean.Engine.DataFeeds.DownloaderDataProvider..ctor() in
/LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Engine/DataFeeds/DownloaderDataProvider.cs:line 57

Martin-Molinero commented 3 months ago

Hey @omidkrad! This issue has been fixed already, please pull the latest lean image 👍

omidkrad commented 2 months ago

Awesome! Thanks, please go ahead and close the issue.

QuantConnect / Lean.Brokerages.InteractiveBrokers