C-SELLERS
commented
3 years ago Hey @ogrande14 , It looks like from the logs that the DebuggerHelper is initializing the wrong debugger. From what I can tell its using the LocalCmdLine debugger. This is the default the DebuggerHelper will initialize if it fails to read the value from config and also the default set in the config.
Do you mind posting your debugging configuration portion of your config file? Check that the "debugging-method" entry is set to "PyCharm". It is case sensitive!
Hope this helps 😄
ogrande14
Hello Guys,
I cloned the Lean Code and got everything set up correctly to run the program on Docker Desktop on my Windows 10. The Backtesting runs fine on Docker and I can also make changes in it. But I can't figure out how to get the Debugger on Pycharm runnig. I followed all the steps in the readme.md and and the Backtesting runs to the end and doesn't stop at the breakpoints. Pycharm doesn't get the connection from the Docker Container as it is permanentely showing "Waiting for Process Connection...".
I tried many things, i.e. adding these lines to the code, which is not described in the readme.md:
import pydevd_pycharm pydevd_pycharm.settrace('localhost', port=6000, stdoutToServer=True, stderrToServer=True)
When I do this, it shows in the Trace that the Connection was refused. After searching for solutions I found that the error message is due to the fact that the docker container is using a different IP than the localhost. But even when I replaced the localhost with the container's IP I get another Connection Error "timed out".
Could anyone explain me how Lean establishes the Connection with Pycharm Debugger while running on a Docker Container? Is there a way I can check this through the console?
Would be glad if anyone can help me. I am planing to use Lean for my trading but I need to backtest and debugg my strategies locally with Pycharm.
Here is the Trace from the console after running with the Backtesting with the configuration for debugging on Pycharm. As I said, I did exactly everything from the readme.md:
20210207 00:43:02.243 Trace:: Using /Lean/Launcher/config.json as configuration file 20210207 00:43:02.364 Trace:: Config.GetValue(): debug-mode - Using default value: False 20210207 00:43:02.365 Trace:: Config.Get(): Configuration key not found. Key: plugin-directory - Using default value: 20210207 00:43:02.367 Trace:: Config.Get(): Configuration key not found. Key: composer-dll-directory - Using default value: /Lean/Launcher/bin/Debug/ 20210207 00:43:02.489 Trace:: Python for .NET Assembly: Python.Runtime, Version=1.0.5.30, Culture=neutral, PublicKeyToken=null 20210207 00:43:02.550 Trace:: Python for .NET Assembly: nPython, Version=1.0.5.30, Culture=neutral, PublicKeyToken=null 20210207 00:43:02.554 Trace:: Config.Get(): Configuration key not found. Key: data-directory - Using default value: ../../../Data/ 20210207 00:43:02.564 Trace:: Config.Get(): Configuration key not found. Key: version-id - Using default value: 20210207 00:43:02.564 Trace:: Config.Get(): Configuration key not found. Key: cache-location - Using default value: /Data 20210207 00:43:02.565 Trace:: Engine.Main(): LEAN ALGORITHMIC TRADING ENGINE v2.4.0.0 Mode: DEBUG (64bit) 20210207 00:43:02.565 Trace:: Engine.Main(): Started 12:43 AM 20210207 00:43:02.591 Trace:: Config.GetValue(): job-project-id - Using default value: 0 20210207 00:43:02.605 Trace:: Config.GetValue(): regression-update-statistics - Using default value: False 20210207 00:43:02.608 Trace:: Config.Get(): Configuration key not found. Key: lean-manager-type - Using default value: LocalLeanManager 20210207 00:43:03.517 Trace:: JobQueue.NextJob(): Selected ../../../Algorithm.Python/AutoRegressiveIntegratedMovingAverageRegressionAlgorithm.py 20210207 00:43:03.536 Trace:: Config.GetValue(): scheduled-event-leaky-bucket-capacity - Using default value: 120 20210207 00:43:03.536 Trace:: Config.GetValue(): scheduled-event-leaky-bucket-time-interval-minutes - Using default value: 1440 20210207 00:43:03.537 Trace:: Config.GetValue(): scheduled-event-leaky-bucket-refill-amount - Using default value: 18 20210207 00:43:03.542 Trace:: Config.Get(): Configuration key not found. Key: algorithm-id - Using default value: AutoRegressiveIntegratedMovingAverageRegressionAlgorithm 20210207 00:43:03.561 Trace:: Config.Get(): Configuration key not found. Key: data-permission-manager - Using default value: DataPermissionManager 20210207 00:43:03.599 Trace:: Config.Get(): Configuration key not found. Key: object-store-root - Using default value: ./storage 20210207 00:43:03.605 Trace:: Config.GetValue(): ignore-version-checks - Using default value: False 20210207 00:43:03.608 Trace:: AlgorithmManager.CreateTokenBucket(): Initializing LeakyBucket: Capacity: 120 RefillAmount: 18 TimeInterval: 1440 20210207 00:43:03.612 Trace:: Config.GetValue(): algorithm-manager-time-loop-maximum - Using default value: 20 20210207 00:43:03.625 Trace:: TextSubscriptionDataSourceReader.SetCacheSize(): Setting cache size to 71582788 items 20210207 00:43:04.261 Trace:: DebuggerHelper.Initialize(): initializing python... 20210207 00:43:04.262 Trace:: PythonInitializer.Initialize(): start... PythonEngine.Initialize(): Runtime.Initialize()... Runtime.Initialize(): Py_Initialize... Runtime.Initialize(): PyEval_InitThreads... Runtime.Initialize(): Initialize types... Runtime.Initialize(): Initialize types end. Runtime.Initialize(): AssemblyManager.Initialize()... Runtime.Initialize(): AssemblyManager.UpdatePath()... PythonEngine.Initialize(): register atexit callback... PythonEngine.Initialize(): GetCLRModule()... PythonEngine.Initialize(): clr GetManifestResourceStream... 20210207 00:43:06.805 Trace:: PythonInitializer.Initialize(): ended 20210207 00:43:06.806 Trace:: DebuggerHelper.Initialize(): python initialization done 20210207 00:43:06.806 Trace:: DebuggerHelper.Initialize(): starting... --Return--
20210207 00:43:10.219 Trace:: BacktestingResultHandler(): Sample Period Set: 122.04 20210207 00:43:10.226 Trace:: Time.TradeableDates(): Security Count: 1 20210207 00:43:10.229 Trace:: Config.GetValue(): forward-console-messages - Using default value: True 20210207 00:43:10.232 Trace:: JOB HANDLERS: 20210207 00:43:10.233 Trace:: DataFeed: QuantConnect.Lean.Engine.DataFeeds.FileSystemDataFeed 20210207 00:43:10.233 Trace:: Setup: QuantConnect.Lean.Engine.Setup.ConsoleSetupHandler 20210207 00:43:10.233 Trace:: RealTime: QuantConnect.Lean.Engine.RealTime.BacktestingRealTimeHandler 20210207 00:43:10.234 Trace:: Results: QuantConnect.Lean.Engine.Results.BacktestingResultHandler 20210207 00:43:10.234 Trace:: Transactions: QuantConnect.Lean.Engine.TransactionHandlers.BacktestingTransactionHandler 20210207 00:43:10.235 Trace:: Alpha: QuantConnect.Lean.Engine.Alphas.DefaultAlphaHandler 20210207 00:43:10.235 Trace:: ObjectStore: QuantConnect.Lean.Engine.Storage.LocalObjectStore 20210207 00:43:10.235 Trace:: History Provider: QuantConnect.Lean.Engine.HistoricalData.SubscriptionDataReaderHistoryProvider 20210207 00:43:10.269 Trace:: Debug: Launching analysis for AutoRegressiveIntegratedMovingAverageRegressionAlgorithm with LEAN Engine v2.4.0.0 20210207 00:43:10.279 Trace:: AlgorithmManager.Run(): Begin DataStream - Start: 1/7/2013 12:00:00 AM Stop: 12/11/2013 11:59:59 PM 20210207 00:43:10.328 Trace:: UniverseSelection.AddPendingInternalDataFeeds(): Adding internal benchmark data feed SPY,SPY,Hour,TradeBar,Trade,Adjusted,Internal 20210207 00:43:10.672 Trace:: Debug: THIS IS A SIMPLE MESSAGE!!!!!
20210207 00:43:10.773 Trace:: Debug: Warning: all market orders sent using daily data, or market orders sent after hours are automatically converted into MarketOnOpen orders. 20210207 00:43:10.933 Trace:: Synchronizer.GetEnumerator(): Exited thread. 20210207 00:43:10.934 Trace:: AlgorithmManager.Run(): Firing On End Of Algorithm... 20210207 00:43:10.936 Trace:: Engine.Run(): Exiting Algorithm Manager 20210207 00:43:10.938 Trace:: FileSystemDataFeed.Exit(): Start. Setting cancellation token... 20210207 00:43:10.940 Trace:: FileSystemDataFeed.Exit(): Exit Finished. 20210207 00:43:10.941 Trace:: DefaultAlphaHandler.Exit(): Exiting... 20210207 00:43:10.946 Trace:: DefaultAlphaHandler.Exit(): Ended 20210207 00:43:10.947 Trace:: BacktestingResultHandler.Exit(): starting... 20210207 00:43:10.948 Trace:: BacktestingResultHandler.Exit(): Saving logs... 20210207 00:43:10.960 Trace:: Debug: Algorithm Id:(AutoRegressiveIntegratedMovingAverageRegressionAlgorithm) completed in 0.70 seconds at 3k data points per second. Processing total of 1,993 data points. 20210207 00:43:10.960 Trace:: Debug: Your log was successfully created and can be retrieved from: /Results/AutoRegressiveIntegratedMovingAverageRegressionAlgorithm-log.txt 20210207 00:43:10.960 Trace:: StopSafely(): waiting for 'Result Thread' thread to stop... 20210207 00:43:10.960 Trace:: BacktestingResultHandler.Run(): Ending Thread... 20210207 00:43:11.291 Trace:: STATISTICS:: Total Trades 65 STATISTICS:: Average Win 0.00% STATISTICS:: Average Loss 0.00% STATISTICS:: Compounding Annual Return 0.145% STATISTICS:: Drawdown 0.100% STATISTICS:: Expectancy 2.190 STATISTICS:: Net Profit 0.134% STATISTICS:: Sharpe Ratio 0.993 STATISTICS:: Probabilistic Sharpe Ratio 49.669% STATISTICS:: Loss Rate 29% STATISTICS:: Win Rate 71% STATISTICS:: Profit-Loss Ratio 3.50 STATISTICS:: Alpha 0.001 STATISTICS:: Beta 0 STATISTICS:: Annual Standard Deviation 0.001 STATISTICS:: Annual Variance 0 STATISTICS:: Information Ratio -2.168 STATISTICS:: Tracking Error 0.099 STATISTICS:: Treynor Ratio -5.187 STATISTICS:: Total Fees $65.00 STATISTICS:: Fitness Score 0 STATISTICS:: Kelly Criterion Estimate 0 STATISTICS:: Kelly Criterion Probability Value 0 STATISTICS:: Sortino Ratio 1.51 STATISTICS:: Return Over Maximum Drawdown 1.819 STATISTICS:: Portfolio Turnover 0 STATISTICS:: Total Insights Generated 0 STATISTICS:: Total Insights Closed 0 STATISTICS:: Total Insights Analysis Completed 0 STATISTICS:: Long Insight Count 0 STATISTICS:: Short Insight Count 0 STATISTICS:: Long/Short Ratio 100% STATISTICS:: Estimated Monthly Alpha Value $0 STATISTICS:: Total Accumulated Estimated Alpha Value $0 STATISTICS:: Mean Population Estimated Insight Value $0 STATISTICS:: Mean Population Direction 0% STATISTICS:: Mean Population Magnitude 0% STATISTICS:: Rolling Averaged Population Direction 0% STATISTICS:: Rolling Averaged Population Magnitude 0% STATISTICS:: OrderListHash 852801548 20210207 00:43:11.292 Trace:: BacktestingResultHandler.SendAnalysisResult(): Processed final packet 20210207 00:43:11.294 Trace:: Engine.Run(): Disconnecting from brokerage... 20210207 00:43:11.295 Trace:: Engine.Run(): Disposing of setup handler... 20210207 00:43:11.299 Trace:: Engine.Main(): Analysis Completed and Results Posted. Engine.Main(): Analysis Complete. 20210207 00:43:11.300 Trace:: Config.GetValue(): close-automatically - Using default value: False Engine.Main(): Press any key to continue. 20210207 00:43:11.302 Trace:: Engine.Main(): Packet removed from queue: AutoRegressiveIntegratedMovingAverageRegressionAlgorithm 20210207 00:43:11.303 Trace:: LeanEngineSystemHandlers.Dispose(): start... 20210207 00:43:11.304 Trace:: LeanEngineSystemHandlers.Dispose(): Disposed of system handlers. 20210207 00:43:11.305 Trace:: LeanEngineAlgorithmHandlers.Dispose(): start... 20210207 00:43:11.307 Trace:: LeanEngineAlgorithmHandlers.Dispose(): Disposed of algorithm handlers. 20210207 00:43:11.312 Trace:: Program.Main(): Exiting Lean... Drücken Sie eine beliebige Taste . . .
Expected Behavior
After setting configuration for debugging-mode with Pycharm and Docker (following the steps in the readme.md for local Delevopment) the Docker Process connects to Pycharm IDE and stops at the specified Breakpoint.
Actual Behavior
Programm runs on Docker Container to the end and doesn't Connect to Pycharm IDE. Pycharm shows "Waiting for process connection...". Debugging is not possible.
Reproducing the Problem
Follow the Steps in the readme.md for Debugging in Pycharm with the Docker Container configuration on Windows 10.
System Information
Windows 10
Checklist
masterbranch