omidkrad
commented
1 month ago ---- REPRO STEPS ---- Using Polygon data downloader fails for querying options quote data. Here's the algo to reproduce:
using System;
using QuantConnect;
using QuantConnect.Algorithm;
using QuantConnect.Brokerages;
using QuantConnect.Data;
public class SampleOptionAlgo : QCAlgorithm
{
public override void Initialize()
{
var endDate = DateTime.Today.AddDays(-1);
var startDate = endDate.AddDays(-3);
SetStartDate(startDate);
SetEndDate(endDate);
SetCash(100000);
SetBrokerageModel(BrokerageName.InteractiveBrokersBrokerage, AccountType.Margin);
var equity = AddEquity("SPY", Resolution.Minute).Symbol;
Securities[equity].SetDataNormalizationMode(DataNormalizationMode.Raw);
SetBenchmark(equity);
var option = AddOption(equity, Resolution.Minute);
option.SetFilter(-5, 5, TimeSpan.FromDays(20), TimeSpan.FromDays(60));
}
public override void OnData(Slice data)
{
foreach (var (symbol, optionChain) in data.OptionChains)
{
foreach (var option in optionChain)
{
Log($"Received {option}");
}
}
}
}(1) I made sure to have correct data subscriptions with Polygon.io and that they are active.
- Polygon Stocks Advanced (includes quotes)
- Polygon Options Advanced (includes quotes)
(4) After setting up Lean CLI to use Polygon, lean.json has the followings set:
"id": "Polygon",
"data-provider": "QuantConnect.Lean.Engine.DataFeeds.DownloaderDataProvider",
"data-downloader": "QuantConnect.Lean.DataSource.Polygon.PolygonDataDownloader",
"polygon-api-key": "<api key>",(3) I run the algo with the following command:
lean backtest SampleOptionAlgo --data-provider-historical "Polygon"Log shows that Advanced Polygon.io subscription plan is needed:
20240721 19:04:51.722 TRACE:: PolygonDataProvider.IsSupported(): Subscription for
Option-QuantConnect.Data.Market.QuoteBar-Quote-Minute will be attempted. An Advanced Polygon.io subscription plan is
required to stream quote data.We have Advanced subscription plan but for some reason it still uses Isolator.ExecuteWithTimeLimit() that I think it's there to delay download requests. The algo fails after 20 minutes.
20240721 19:24:20.862 ERROR:: Security.ExecuteWithTimeLimit(): Algorithm took longer than 20 minutes on a
single time loop. CurrentTimeStepElapsed: 20.0 minutesLog shows that the stock data for the last 3 days get downloaded but option data fails to download. Also can confirm that from the downloaded files in the data folder.
From the logs:
DATA USAGE:: Total data requests 10
DATA USAGE:: Succeeded data requests 3
DATA USAGE:: Failed data requests 7
Martin-Molinero
jhonabreul
Expected Behavior
Able to download options quote data with Polygon
Actual Behavior
Polygon downloader waits for extended time for quote data which causes the runtime to time out and exit.
Potential Solution
Review and address
Reproducing the Problem
See details in the next post below.
System Information
Windows 10
Checklist
masterbranch