QuantConnect / Lean

Lean Algorithmic Trading Engine by QuantConnect (Python, C#)
https://lean.io
Apache License 2.0
9.65k stars 3.24k forks source link

SubscriptionDataSource.Rest Polling Daily for Hourly Resolution #4527

Closed AmazingAbhi closed 1 year ago

AmazingAbhi commented 4 years ago

Expected Behavior

The OnData event should Poll the GetSource Function every hour in case SubscriptionDataSource.Rest api is used for getting the data.

Actual Behavior

Even after specifying, minute or hour as resolution, the URL is being polled daily instead due to which the daily bars are going in OnData.

Potential Solution

Based on the resolution specified for the universe the SubscriptionDataSource.Rest api should be polled every resolution time step.

Reproducing the Problem

Following is the Custom Data using algorithm. If you add a breakpoint at return of GetSource, You will notice the execution is coming at it on daily level rather than Hourly frequency

`class CustomDataStock(QCAlgorithm):

def Initialize(self):
    self.SetStartDate(2020, 1, 1)
    self.SetEndDate(2020, 1, 31)
    self.SetCash(100000)
    self.reso = Resolution.Hour
    self.UniverseSettings.Resolution = self.reso

    # Define the symbol and "type" of our generic data:
    self.appleEquity = self.AddEquity("APPL", self.reso, Market.NSE,True,1,False).Symbol
    self.apple = self.AddData(CustomStock, self.appleEquity, self.reso).Symbol
    self.prices = []

class Stock(PythonData):

def GetSource(self, config, date, isLiveMode):
     return SubscriptionDataSource(<URL>, SubscriptionTransportMedium.Rest)

`

System Information

Windows 10 Python 3.6.8

Checklist

Martin-Molinero commented 4 years ago

Hey @abhi555shek! I'm supposing this is a live trading issue, please correct me if wrong. Can't seem to reproduce it, tested using Resolution.Minute and GetSouce and the rest call are triggered every minute, note that the date sent in to the GetSource method is just the date component (minutes/hours/etc are not set). The following is the algorithm I used:

def Initialize(self):
    self.SetStartDate(2020, 1, 1)
    self.SetEndDate(2020, 1, 31)
    self.reso = Resolution.Minute
    self.UniverseSettings.Resolution = self.reso

    # Define the symbol and "type" of our generic data:
    self.appleEquity = self.AddEquity("APPL", self.reso, Market.USA,True,1,False).Symbol
    self.apple = self.AddData(CustomStock, self.appleEquity, self.reso).Symbol
    self.prices = []
class CustomStock(PythonData):
def GetSource(self, config, date, isLiveMode):
     return SubscriptionDataSource("https://www.SomeAddress.com", SubscriptionTransportMedium.Rest)

def Reader(self, config, line, date, isLiveMode):
    return None
Martin-Molinero commented 4 years ago

Note that for backtesting the behavior is different: it will only make a new request when the source address has changed in a daily manner, maybe this could explain the behavior seen? This is currently by design, for each tradable date it will refresh the source and make the rest request if the source address has changed

AmazingAbhi commented 4 years ago

Martin, I am using the api for backtesting purpose using the database. The api address is like "http://xxx.xx.xx.xx:5342/" +"symbol"/"timestamp". So for every iteration via GetSource, the url will change as per the timestamp but the issue is the execution increases the date everytime it comes into the GetSource even for minute and hourly resolutions.

For e.g. Resolution = Hour Iteration 1: Date = (2020,1,1) Iteration 2: Date = (2020,1,2) (I expected the second iteration at same date (2020,1,1) with time increased by one hour as this is the resolution time step and api should provide only one data point)

petioptrv commented 4 years ago

I second this problem ^

danyg commented 2 years ago

I found a similar problem using self.UniverseSettings.Resolution = Resolution.Hour

The following code reproduce the error using LEAN locally and does not happen using quantconnect backtest nodes Backtest with expected behaviour

Code to reproduce (also can be found in quantconnect backtest)

from AlgorithmImports import *

class Bugreport(QCAlgorithm):
    _ondatacalls = 0

    def Initialize(self):
        self.SetStartDate(2021, 4, 1)  # Set Start Date
        self.SetEndDate(2021, 5, 1)  # Set End Date
        self.SetCash(100000)  # Set Strategy Cash

        self._selection = []

        self.UniverseSettings.Resolution = Resolution.Hour
        self.AddUniverse(self._coarse_selection)

    def _coarse_selection(self, coarse):
        if len(self._selection) == 0:
            self._selection = [Symbol.Create(
                "BTCUSD", SecurityType.Crypto, Market.Bitfinex)]
        return self._selection

    def OnData(self, data):
        """OnData event is the primary entry point for your algorithm. Each new data point will be pumped in here.
            Arguments:
                data: Slice object keyed by symbol containing the stock data
        """
        self._ondatacalls += 1
        self.Debug("On Data " + str(self._ondatacalls))
        if not self.Portfolio.Invested:
            each_percent = 1/len(self._selection)
            for symbol in self._selection:
                self.SetHoldings(symbol, each_percent)
            self.Debug("Purchased Stock")
        if self.Portfolio.Invested:
            self.Liquidate()
            self.Debug("Sold Stock")

Log generated by executing this with lean, version 1.0.71

2021-11-16T10:53:23.8022580Z TRACE:: Config.Get(): Configuration key not found. Key: data-directory - Using default value: ../../../Data/
2021-11-16T10:53:23.8213624Z TRACE:: Config.Get(): Configuration key not found. Key: version-id - Using default value: 
2021-11-16T10:53:23.8218257Z TRACE:: Config.Get(): Configuration key not found. Key: cache-location - Using default value: /Lean/Data
2021-11-16T10:53:23.8225670Z TRACE:: Engine.Main(): LEAN ALGORITHMIC TRADING ENGINE v2.5.0.0 Mode: DEBUG (64bit) Host: Daniel
2021-11-16T10:53:23.8474271Z TRACE:: Engine.Main(): Started 10:53 AM
2021-11-16T10:53:23.8575641Z TRACE:: Config.Get(): Configuration key not found. Key: lean-manager-type - Using default value: LocalLeanManager
2021-11-16T10:53:23.9018833Z TRACE:: JobQueue.NextJob(): Selected /LeanCLI/main.py
2021-11-16T10:53:24.0846459Z TRACE:: Config.GetValue(): scheduled-event-leaky-bucket-capacity - Using default value: 120
2021-11-16T10:53:24.0851892Z TRACE:: Config.GetValue(): scheduled-event-leaky-bucket-time-interval-minutes - Using default value: 1440
2021-11-16T10:53:24.0854539Z TRACE:: Config.GetValue(): scheduled-event-leaky-bucket-refill-amount - Using default value: 18
2021-11-16T10:53:24.1030477Z TRACE:: Config.Get(): Configuration key not found. Key: job-organization-id - Using default value: 
2021-11-16T10:53:24.1039909Z TRACE:: Config.Get(): Configuration key not found. Key: data-permission-manager - Using default value: DataPermissionManager
2021-11-16T10:53:24.1450094Z TRACE:: AlgorithmManager.CreateTokenBucket(): Initializing LeakyBucket: Capacity: 120 RefillAmount: 18 TimeInterval: 1440
2021-11-16T10:53:24.1535518Z TRACE:: Config.GetValue(): algorithm-manager-time-loop-maximum - Using default value: 20
2021-11-16T10:53:24.2251567Z TRACE:: TextSubscriptionDataSourceReader.SetCacheSize(): Setting cache size to 71582788 items
2021-11-16T10:53:24.7306008Z TRACE:: Config.GetValue(): algorithm-creation-timeout - Using default value: 90
2021-11-16T10:53:24.7367156Z TRACE:: PythonInitializer.Initialize(): start...
2021-11-16T10:53:26.5212960Z TRACE:: PythonInitializer.Initialize(): ended
2021-11-16T10:53:26.5268363Z TRACE:: AlgorithmPythonWrapper(): Python version 3.6.8 |Anaconda, Inc.| (default, Dec 30 2018, 01:25:33) 
[GCC 7.3.0]: Importing python module main
2021-11-16T10:53:29.2271960Z TRACE:: AlgorithmPythonWrapper(): main successfully imported.
2021-11-16T10:53:29.2307767Z TRACE:: AlgorithmPythonWrapper(): Creating IAlgorithm instance.
2021-11-16T10:53:29.2400971Z TRACE:: Config.GetValue(): api-data-update-period - Using default value: 1
2021-11-16T10:53:29.5477663Z TRACE:: Config.GetValue(): mute-python-library-logging - Using default value: True
2021-11-16T10:53:29.5907907Z TRACE:: LocalObjectStore.Initialize(): Storage Root: /Storage/QCAlgorithm. StorageFileCount 100. StorageLimitMB 5
2021-11-16T10:53:29.6224006Z TRACE:: BacktestingSetupHandler.Setup(): Setting up job: Plan: Free, UID: 171157, PID: 814184126, Version: 2.5.0.0, Source: WebIDE
2021-11-16T10:53:29.6341580Z TRACE:: Config.Get(): Configuration key not found. Key: security-data-feeds - Using default value: 
2021-11-16T10:53:29.8453817Z TRACE:: Config.GetValue(): data-feed-max-work-weight - Using default value: 400
2021-11-16T10:53:29.8466594Z TRACE:: Config.GetValue(): data-feed-workers-count - Using default value: 12
2021-11-16T10:53:29.8473210Z TRACE:: WeightedWorkScheduler(): will use 12 workers and MaxWorkWeight is 400
2021-11-16T10:53:29.8487892Z DEBUG:: DataManager.AddSubscription(): Added QC-UNIVERSE-COARSE-USA-6FD84044-7580-41B7-B9DD-5FAECE9386EA,QC-UNIVERSE-COARSE-USA-6FD84044-7580-41B7-B9DD-5FAECE9386EA,Daily,CoarseFundamental,Trade,Adjusted,Internal. Start: 4/1/2021 4:00:00 AM. End: 5/2/2021 3:59:59 AM
2021-11-16T10:53:29.8704312Z TRACE:: Config.GetValue(): show-missing-data-logs - Using default value: False
2021-11-16T10:53:29.8890086Z TRACE:: BaseSetupHandler.SetupCurrencyConversions():
Symbol      Quantity    Conversion = Value in USD
USD: $      100000.00 @       1.00 = $100000.0
-------------------------------------------------
CashBook Total Value:                $100000.0

2021-11-16T10:53:29.8990856Z TRACE:: SetUp Backtesting: User: 171157 ProjectId: 814184126 AlgoId: 1713821531
2021-11-16T10:53:29.8997814Z TRACE:: Dates: Start: 04/01/2021 End: 05/01/2021 Cash: ¤100,000.00
2021-11-16T10:53:29.9036455Z TRACE:: BacktestingResultHandler(): Sample Period Set: 11.16
2021-11-16T10:53:29.9135501Z TRACE:: Config.GetValue(): forward-console-messages - Using default value: True
2021-11-16T10:53:29.9171060Z TRACE:: JOB HANDLERS: 
2021-11-16T10:53:29.9180718Z TRACE::          DataFeed:     QuantConnect.Lean.Engine.DataFeeds.FileSystemDataFeed
2021-11-16T10:53:29.9190486Z TRACE::          Setup:        QuantConnect.Lean.Engine.Setup.ConsoleSetupHandler
2021-11-16T10:53:29.9194049Z TRACE::          RealTime:     QuantConnect.Lean.Engine.RealTime.BacktestingRealTimeHandler
2021-11-16T10:53:29.9220354Z TRACE::          Results:      QuantConnect.Lean.Engine.Results.BacktestingResultHandler
2021-11-16T10:53:29.9225877Z TRACE::          Transactions: QuantConnect.Lean.Engine.TransactionHandlers.BacktestingTransactionHandler
2021-11-16T10:53:29.9230747Z TRACE::          Alpha:        QuantConnect.Lean.Engine.Alphas.DefaultAlphaHandler
2021-11-16T10:53:29.9238488Z TRACE::          ObjectStore:  QuantConnect.Lean.Engine.Storage.LocalObjectStore
2021-11-16T10:53:29.9240831Z TRACE::          History Provider:     QuantConnect.Lean.Engine.HistoricalData.SubscriptionDataReaderHistoryProvider
2021-11-16T10:53:29.9713727Z TRACE:: Event Name "Daily Sampling", scheduled to run at 4/1/2021 4:00:00 AM (UTC)...
2021-11-16T10:53:29.9736854Z TRACE:: AlgorithmManager.Run(): Begin DataStream - Start: 4/1/2021 12:00:00 AM Stop: 5/1/2021 11:59:59 PM
2021-11-16T10:53:29.9805053Z TRACE:: Debug: Launching analysis for 1713821531 with LEAN Engine v2.5.0.0
2021-11-16T10:53:30.0739610Z DEBUG:: DataManager.SubscriptionManagerGetOrAdd(): subscription already added: BTCUSD,BTCUSD,Hour,TradeBar,Trade,Adjusted
2021-11-16T10:53:30.0831640Z DEBUG:: DataManager.SubscriptionManagerGetOrAdd(): subscription already added: BTCUSD,BTCUSD,Hour,QuoteBar,Quote,Adjusted
2021-11-16T10:53:30.0921373Z DEBUG:: DataManager.AddSubscription(): Added BTCUSD,BTCUSD,Hour,TradeBar,Trade,Adjusted. Start: 4/1/2021 4:00:00 AM. End: 5/2/2021 3:59:59 AM
2021-11-16T10:53:30.0950839Z DEBUG:: DataManager.AddSubscription(): Added BTCUSD,BTCUSD,Hour,QuoteBar,Quote,Adjusted. Start: 4/1/2021 4:00:00 AM. End: 5/2/2021 3:59:59 AM
2021-11-16T10:53:30.1312651Z DEBUG:: UniverseSelection.ApplyUniverseSelection(): 4/1/2021 4:00:00 AM: SecurityChanges:  Added: BTCUSD E3
2021-11-16T10:53:30.1411722Z DEBUG:: DataManager.RemoveSubscription(): Removed BTCUSD,BTCUSD,Hour,TradeBar,Trade,Adjusted
2021-11-16T10:53:30.1418767Z DEBUG:: Synchronizer.SubscriptionFinished(): Finished subscription:BTCUSD,BTCUSD,Hour,TradeBar,Trade,Adjusted at 4/1/2021 4:00:00 AM UTC
2021-11-16T10:53:30.1424380Z DEBUG:: DataManager.RemoveSubscription(): Removed BTCUSD,BTCUSD,Hour,QuoteBar,Quote,Adjusted
2021-11-16T10:53:30.1428895Z DEBUG:: Synchronizer.SubscriptionFinished(): Finished subscription:BTCUSD,BTCUSD,Hour,QuoteBar,Quote,Adjusted at 4/1/2021 4:00:00 AM UTC
2021-11-16T10:53:30.1784255Z DEBUG:: DataManager.AddSubscription(): Added SPY,SPY,Hour,TradeBar,Trade,Adjusted,Internal. Start: 3/31/2021 4:00:00 AM. End: 5/2/2021 3:59:59 AM
2021-11-16T10:53:30.1826009Z TRACE:: UniverseSelection.AddPendingInternalDataFeeds(): Adding internal benchmark data feed SPY,SPY,Hour,TradeBar,Trade,Adjusted,Internal
2021-11-16T10:53:30.5014442Z DEBUG:: DataManager.RemoveSubscription(): Removed SPY,SPY,Hour,TradeBar,Trade,Adjusted,Internal
2021-11-16T10:53:30.5028571Z DEBUG:: Synchronizer.SubscriptionFinished(): Finished subscription:SPY,SPY,Hour,TradeBar,Trade,Adjusted,Internal at 4/1/2021 4:00:00 AM UTC
2021-11-16T10:53:30.5549958Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-01 04:00:00 UTC Scheduled at 2021-04-01 04:00:00 UTC
2021-11-16T10:53:30.5590425Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.5673383Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-02 04:00:00 UTC
2021-11-16T10:53:30.6139361Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-02 04:00:00 UTC Scheduled at 2021-04-02 04:00:00 UTC
2021-11-16T10:53:30.6146712Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.6210677Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-03 04:00:00 UTC
2021-11-16T10:53:30.6312150Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-03 04:00:00 UTC Scheduled at 2021-04-03 04:00:00 UTC
2021-11-16T10:53:30.6320185Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.6323658Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-04 04:00:00 UTC
2021-11-16T10:53:30.6327402Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-04 04:00:00 UTC Scheduled at 2021-04-04 04:00:00 UTC
2021-11-16T10:53:30.6330621Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.6341979Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-05 04:00:00 UTC
2021-11-16T10:53:30.6352116Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-05 04:00:00 UTC Scheduled at 2021-04-05 04:00:00 UTC
2021-11-16T10:53:30.6426279Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.6432916Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-06 04:00:00 UTC
2021-11-16T10:53:30.6487686Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-06 04:00:00 UTC Scheduled at 2021-04-06 04:00:00 UTC
2021-11-16T10:53:30.6493025Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.6500241Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-07 04:00:00 UTC
2021-11-16T10:53:30.6509224Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-07 04:00:00 UTC Scheduled at 2021-04-07 04:00:00 UTC
2021-11-16T10:53:30.6534755Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.6580361Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-08 04:00:00 UTC
2021-11-16T10:53:30.6586570Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-08 04:00:00 UTC Scheduled at 2021-04-08 04:00:00 UTC
2021-11-16T10:53:30.6610222Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.6627067Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-09 04:00:00 UTC
2021-11-16T10:53:30.6632532Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-09 04:00:00 UTC Scheduled at 2021-04-09 04:00:00 UTC
2021-11-16T10:53:30.6635761Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.6638459Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-10 04:00:00 UTC
2021-11-16T10:53:30.6664221Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-10 04:00:00 UTC Scheduled at 2021-04-10 04:00:00 UTC
2021-11-16T10:53:30.6686626Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.6703555Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-11 04:00:00 UTC
2021-11-16T10:53:30.6709043Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-11 04:00:00 UTC Scheduled at 2021-04-11 04:00:00 UTC
2021-11-16T10:53:30.6716868Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.6722222Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-12 04:00:00 UTC
2021-11-16T10:53:30.6725405Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-12 04:00:00 UTC Scheduled at 2021-04-12 04:00:00 UTC
2021-11-16T10:53:30.6764323Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.6793107Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-13 04:00:00 UTC
2021-11-16T10:53:30.6840651Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-13 04:00:00 UTC Scheduled at 2021-04-13 04:00:00 UTC
2021-11-16T10:53:30.6845128Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.6855707Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-14 04:00:00 UTC
2021-11-16T10:53:30.6861299Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-14 04:00:00 UTC Scheduled at 2021-04-14 04:00:00 UTC
2021-11-16T10:53:30.6864573Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.6868261Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-15 04:00:00 UTC
2021-11-16T10:53:30.6876186Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-15 04:00:00 UTC Scheduled at 2021-04-15 04:00:00 UTC
2021-11-16T10:53:30.6879042Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.6883413Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-16 04:00:00 UTC
2021-11-16T10:53:30.6888824Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-16 04:00:00 UTC Scheduled at 2021-04-16 04:00:00 UTC
2021-11-16T10:53:30.6891966Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.6895402Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-17 04:00:00 UTC
2021-11-16T10:53:30.6899315Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-17 04:00:00 UTC Scheduled at 2021-04-17 04:00:00 UTC
2021-11-16T10:53:30.6931575Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.6974899Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-18 04:00:00 UTC
2021-11-16T10:53:30.6980202Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-18 04:00:00 UTC Scheduled at 2021-04-18 04:00:00 UTC
2021-11-16T10:53:30.7017156Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.7021361Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-19 04:00:00 UTC
2021-11-16T10:53:30.7024888Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-19 04:00:00 UTC Scheduled at 2021-04-19 04:00:00 UTC
2021-11-16T10:53:30.7060529Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.7063887Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-20 04:00:00 UTC
2021-11-16T10:53:30.7068549Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-20 04:00:00 UTC Scheduled at 2021-04-20 04:00:00 UTC
2021-11-16T10:53:30.7070771Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.7073381Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-21 04:00:00 UTC
2021-11-16T10:53:30.7101048Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-21 04:00:00 UTC Scheduled at 2021-04-21 04:00:00 UTC
2021-11-16T10:53:30.7123875Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.7127600Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-22 04:00:00 UTC
2021-11-16T10:53:30.7149493Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-22 04:00:00 UTC Scheduled at 2021-04-22 04:00:00 UTC
2021-11-16T10:53:30.7154456Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.7164993Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-23 04:00:00 UTC
2021-11-16T10:53:30.7171353Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-23 04:00:00 UTC Scheduled at 2021-04-23 04:00:00 UTC
2021-11-16T10:53:30.7175400Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.7180242Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-24 04:00:00 UTC
2021-11-16T10:53:30.7184909Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-24 04:00:00 UTC Scheduled at 2021-04-24 04:00:00 UTC
2021-11-16T10:53:30.7189009Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.7192247Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-25 04:00:00 UTC
2021-11-16T10:53:30.7195702Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-25 04:00:00 UTC Scheduled at 2021-04-25 04:00:00 UTC
2021-11-16T10:53:30.7199427Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.7205144Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-26 04:00:00 UTC
2021-11-16T10:53:30.7215074Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-26 04:00:00 UTC Scheduled at 2021-04-26 04:00:00 UTC
2021-11-16T10:53:30.7217725Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.7220986Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-27 04:00:00 UTC
2021-11-16T10:53:30.7225158Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-27 04:00:00 UTC Scheduled at 2021-04-27 04:00:00 UTC
2021-11-16T10:53:30.7227685Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.7231865Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-28 04:00:00 UTC
2021-11-16T10:53:30.7238322Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-28 04:00:00 UTC Scheduled at 2021-04-28 04:00:00 UTC
2021-11-16T10:53:30.7242943Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.7251751Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-29 04:00:00 UTC
2021-11-16T10:53:30.7282948Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-29 04:00:00 UTC Scheduled at 2021-04-29 04:00:00 UTC
2021-11-16T10:53:30.7305429Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.7310763Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-04-30 04:00:00 UTC
2021-11-16T10:53:30.7319471Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-04-30 04:00:00 UTC Scheduled at 2021-04-30 04:00:00 UTC
2021-11-16T10:53:30.7323835Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.7327318Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-05-01 04:00:00 UTC
2021-11-16T10:53:30.7333391Z DEBUG:: DataManager.RemoveSubscription(): Removed QC-UNIVERSE-COARSE-USA-6FD84044-7580-41B7-B9DD-5FAECE9386EA,QC-UNIVERSE-COARSE-USA-6FD84044-7580-41B7-B9DD-5FAECE9386EA,Daily,CoarseFundamental,Trade,Adjusted,Internal
2021-11-16T10:53:30.7371761Z DEBUG:: Synchronizer.SubscriptionFinished(): Finished subscription:QC-UNIVERSE-COARSE-USA-6FD84044-7580-41B7-B9DD-5FAECE9386EA,QC-UNIVERSE-COARSE-USA-6FD84044-7580-41B7-B9DD-5FAECE9386EA,Daily,CoarseFundamental,Trade,Adjusted,Internal at 5/1/2021 4:00:00 AM UTC
2021-11-16T10:53:30.7382040Z TRACE:: ScheduledEvent.Daily Sampling: Firing at 2021-05-01 04:00:00 UTC Scheduled at 2021-05-01 04:00:00 UTC
2021-11-16T10:53:30.7390493Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.7420515Z TRACE:: ScheduledEvent.Daily Sampling: Next event: 2021-05-02 04:00:00 UTC
2021-11-16T10:53:30.7428184Z TRACE:: Synchronizer.GetEnumerator(): Exited thread.
2021-11-16T10:53:30.7443985Z TRACE:: AlgorithmManager.Run(): Firing On End Of Algorithm...
2021-11-16T10:53:30.7539936Z DEBUG:: BaseResultsHandler.SamplePerformance(): 4:00 AM >0
2021-11-16T10:53:30.7544283Z TRACE:: Engine.Run(): Exiting Algorithm Manager
2021-11-16T10:53:30.7565910Z TRACE:: FileSystemDataFeed.Exit(): Start. Setting cancellation token...
2021-11-16T10:53:30.7639486Z TRACE:: FileSystemDataFeed.Exit(): Exit Finished.
2021-11-16T10:53:30.7646428Z TRACE:: DefaultAlphaHandler.Exit(): Exiting...
2021-11-16T10:53:30.7735932Z TRACE:: DefaultAlphaHandler.Exit(): Ended
2021-11-16T10:53:30.7742672Z TRACE:: BacktestingResultHandler.Exit(): starting...
2021-11-16T10:53:30.7748622Z TRACE:: BacktestingResultHandler.Exit(): Saving logs...
2021-11-16T10:53:30.7839639Z TRACE:: Debug: Algorithm Id:(1713821531) completed in 0.83 seconds at 0k data points per second. Processing total of 7 data points.
2021-11-16T10:53:30.7912421Z TRACE:: Debug: Your log was successfully created and can be retrieved from: /Results/1713821531-log.txt
2021-11-16T10:53:30.7915010Z TRACE:: StopSafely(): waiting for 'Result Thread' thread to stop...
2021-11-16T10:53:30.7918000Z TRACE:: BacktestingResultHandler.Run(): Ending Thread...
2021-11-16T10:53:31.0060958Z TRACE:: Config.GetValue(): regression-update-statistics - Using default value: False
2021-11-16T10:53:31.0068640Z TRACE:: 
STATISTICS:: Total Trades 0
STATISTICS:: Average Win 0%
STATISTICS:: Average Loss 0%
STATISTICS:: Compounding Annual Return 0%
STATISTICS:: Drawdown 0%
STATISTICS:: Expectancy 0
STATISTICS:: Net Profit 0%
STATISTICS:: Sharpe Ratio 0
STATISTICS:: Probabilistic Sharpe Ratio 0%
STATISTICS:: Loss Rate 0%
STATISTICS:: Win Rate 0%
STATISTICS:: Profit-Loss Ratio 0
STATISTICS:: Alpha 0
STATISTICS:: Beta 0
STATISTICS:: Annual Standard Deviation 0
STATISTICS:: Annual Variance 0
STATISTICS:: Information Ratio 0
STATISTICS:: Tracking Error 0
STATISTICS:: Treynor Ratio 0
STATISTICS:: Total Fees $0.00
STATISTICS:: Estimated Strategy Capacity $0
STATISTICS:: Lowest Capacity Asset 
STATISTICS:: Fitness Score 0
STATISTICS:: Kelly Criterion Estimate 0
STATISTICS:: Kelly Criterion Probability Value 0
STATISTICS:: Sortino Ratio 79228162514264337593543950335
STATISTICS:: Return Over Maximum Drawdown 79228162514264337593543950335
STATISTICS:: Portfolio Turnover 0
STATISTICS:: Total Insights Generated 0
STATISTICS:: Total Insights Closed 0
STATISTICS:: Total Insights Analysis Completed 0
STATISTICS:: Long Insight Count 0
STATISTICS:: Short Insight Count 0
STATISTICS:: Long/Short Ratio 100%
STATISTICS:: Estimated Monthly Alpha Value $0
STATISTICS:: Total Accumulated Estimated Alpha Value $0
STATISTICS:: Mean Population Estimated Insight Value $0
STATISTICS:: Mean Population Direction 0%
STATISTICS:: Mean Population Magnitude 0%
STATISTICS:: Rolling Averaged Population Direction 0%
STATISTICS:: Rolling Averaged Population Magnitude 0%
STATISTICS:: OrderListHash d41d8cd98f00b204e9800998ecf8427e
2021-11-16T10:53:31.0071573Z TRACE:: BacktestingResultHandler.SendAnalysisResult(): Processed final packet
2021-11-16T10:53:31.0077223Z TRACE:: Engine.Run(): Disconnecting from brokerage...
2021-11-16T10:53:31.0104903Z TRACE:: Engine.Run(): Disposing of setup handler...
2021-11-16T10:53:31.0111220Z TRACE:: Engine.Main(): Analysis Completed and Results Posted.
2021-11-16T10:53:31.0119918Z TRACE:: Engine.Main(): Packet removed from queue: 1713821531
2021-11-16T10:53:31.0124962Z TRACE:: LeanEngineSystemHandlers.Dispose(): start...
2021-11-16T10:53:31.0131715Z TRACE:: LeanEngineSystemHandlers.Dispose(): Disposed of system handlers.
2021-11-16T10:53:31.0138802Z TRACE:: LeanEngineAlgorithmHandlers.Dispose(): start...
2021-11-16T10:53:31.0288538Z TRACE:: LeanEngineAlgorithmHandlers.Dispose(): Disposed of algorithm handlers.

As you can see in the logs there is no On Data debug logs, nor the algorithm produce any order where the code is buying at the beginning of an hour an selling in the next, Using crypto which is near immediate

Using AddUniverseSelection the issue cannot be reproduced, seems to be only happening when using Resolution.Hour with AddUniverse on local

HTH

jaredbroad commented 1 year ago

Closing as we could not repeat the issue