Python usage on Linux

[Kyxsune]

Hey sorry If I am missing something,

but I am having trouble getting Lean to use the Python.dll on linux. The current documentation points to using the .exe file in the debug folder, (which has its own config). However changing the config file in this folder only leads to errors as the neccessary python files are not in the debug file.

I think I am missing something. Any advice?

[AlexCatarino]

First, you have to execute this bash script. It will create the QuantConnect.Algorithm.Python.dll and copy it in the Launcher/bin/Debug. Then you need to change config.json:

  "algorithm-language": "Python",
  "algorithm-location": "QuantConnect.Algorithm.Python.dll",

Could you please consider beta testing our new python framework (pythonnet branch)?

With this new framework, we can import third party libraries (Ref,: #525).

[jaredbroad]

@Kyxsune I wouldn't recommend getting too familiar with the previous branch. The new system is quite a lot more powerful. Closing as Alex sovled above.

[Kyxsune]

@jaredbroad ok, so I cloned the new branch and edited the bash script to create the dll files for the debug folder. I understand to backtest the code I have to run it from inside the Debug folder. My only question at this point, is do I have to create a new dll file with each edit to the algorithm? I think so, but I want to confirm.

Also, this is only compatible with 2.7 packages (obviously, but worth confirming)?

[Kyxsune]

@AlexCatarino not sure where to go for the beta test. But I am having some trouble getting the import to work. I am testing just numpy and pandas.

20170118 22:50:21 Trace:: Loader.TryCreatePythonAlgorithm(): Importing python module... 20170118 22:50:21 ERROR:: Loader.TryCreatePythonAlgorithm(): IronPython.Runtime.Exceptions.ImportException: No module named numpy at Microsoft.Scripting.Runtime.LightExceptions.CheckAndThrow (System.Object value) [0x0000a] in <ea56ddf231e34894a5190fc7cc35bc10>:0 at DLRCachedCode.Main$17 (IronPython.Runtime.CodeContext $globalContext, IronPython.Runtime.FunctionCode $functionCode) [0x00b06] in <3ea92402ea5f4a85bc61b00a1f2d2a90>:0 20170118 22:50:21 ERROR:: Engine.Run(): System.Exception: No module named numpy - could not locate 'main' module. Please make sure you have a main.py file in your project.: try re-building algorithm. at QuantConnect.Lean.Engine.Setup.ConsoleSetupHandler.CreateAlgorithmInstance (System.String assemblyPath, QuantConnect.Language language) [0x0005c] in <6fb4435fe9f940a2bb8fe1c1c546decf>:0 at QuantConnect.Lean.Engine.Engine.Run (QuantConnect.Packets.AlgorithmNodePacket job, System.String assemblyPath) [0x00130] in <6fb4435fe9f940a2bb8fe1c1c546decf>:0

I have made sure that numpy works with an import statement with my python interpreter (both 3.4 and 2.7)

I'm probably missing something again

[AlexCatarino]

@Kyxsune, with the new framework, we do not need to execute the bash script, since it is a procedure required for IronPython. Sorry that I haven't made it clear. Looking at your stack, I can tell that you not working with the new framework (IronPython.Runtime.Exceptions.ImportException). With the old framework, you cannot import numpy.

Here are the steps for running the new branch:

$ git clone https://github.com/QuantConnect/Lean.git
$ cd Lean
$ git checkout pythonnet
$ sed -i 's/4.5/4.0/' Algorithm.VisualBasic/QuantConnect.Algorithm.VisualBasic.vbproj
$ sed -i -e 's/UCS2/UCS4/' Algorithm/QuantConnect.Algorithm.csproj
$ sed -i -e 's/UCS2/UCS4/' AlgorithmFactory/QuantConnect.AlgorithmFactory.csproj
$ wget https://nuget.org/nuget.exe
$ mono nuget.exe restore QuantConnect.Lean.sln
$ xbuild
$ cd Lean/Launcher/bin/Debug`

Edit the config.json file: "algorithm-language": "Python",

Run the executable: $ mono QuantConnect.Lean.Launcher.exe

And it will run the BasicTemplateAlgorithn.py algorithm.

[Kyxsune]

@AlexCatarino Thanks a ton for the explicit instructions along with the clarification.

It successfully imports pandas and numpy. With only a single non fatal error:

20170119 08:31:46 ERROR:: Failed to get StreamReader for data source(../../../Data/equity/usa/minute/spy/20131004_trade.zip), symbol(SPY). Skipping date(10/4/2013). Reader is null.

Aside from adding the line to check pandas to the BasicTemplateAlgorthn.py I have made no other edits. I will test other packages. Tensorflow/Tflearn throws an error, though it appears to import correctly. Where should I leave the testing feedback?

[den-run-ai]

@Kyxsune regarding tensorflow see this issue, it works now in the master of pythonnet!

https://github.com/pythonnet/pythonnet/issues/299

[den-run-ai]

is Failed to get StreamReader for data reported from pythonnet or lean?

[jaredbroad]

LEAN; new code seeds the security so new assets always start with a price (defaulting to the last tick, so day before start of backtest). Its a non-fatal error and will revert to previous behavior when no data found (price set to $0 at start).