ScottfreeLLC / AlphaPy

Python AutoML for Trading Systems and Sports Betting
Apache License 2.0
1.11k stars 201 forks source link

Startup problems #21

Closed roylend closed 5 years ago

roylend commented 6 years ago

Hello Scott,

I like this project a lot but I have issues getting it up and running. I tried both under Windows and Ubuntu. In Ubuntu at least the pip3 install alphapy worked fine.

But when I move to one of the example directories and try to run the commands you show in the documentation, it just does not find the alphapy or market_flow files. Even when I put those files inside the same directory and try to run them from the Python editor environment, I get all kinds of errors.

How can you run the examples directly from the Python editor environment?

mrconway commented 6 years ago

Could you post some error logs here? Thanks.

roylend commented 6 years ago

Hello Mark,

My first issue was with pandas. It appears the library is very reliant on having the right version of Pandas (not the last one). I managed to solve this one by trying all Pandas versions between 1.19 and 2.22. When using 2.22 it worked.

The issue I now have is with installing tkinter within the Ubuntu environment. When I run marketflow, I get this tkinter issue. However, when I try to install tkinter for Python3, I get 404 errors from the Ubuntu archive.

Let’s hope I manage to solve these installation issues. One question about the library itself. I recently saw from academic research that AI models trained on Japanese stocks for example performed equally well on US stocks as compared to AI models trained on US stocks. So can I use Alphapy to train the algorithm on let’s say the history of 500 stocks and then use this complete trained model to predict on all invididual stocks? Instead of training only on the history of the specific stock you are trying to predict?

Roy

Van: Mark Conway [mailto:notifications@github.com] Verzonden: maandag 4 juni 2018 1:52 Aan: ScottFreeLLC/AlphaPy AlphaPy@noreply.github.com CC: roylend roy.lenders@hotmail.com; Author author@noreply.github.com Onderwerp: Re: [ScottFreeLLC/AlphaPy] Startup problems (#21)

Could you post some error logs here? Thanks.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/ScottFreeLLC/AlphaPy/issues/21#issuecomment-394201993, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AmCbhSEWEhTfevLARNHV9i2bARdcFnpIks5t5HahgaJpZM4UYM34.

roylend commented 6 years ago

Mark,

Managed to fix the tkinter issue as well by updating the URL’s for the Ubuntu environment to “old-releases”.

Now when I run marketflow or “main” in the directory with the Trading example. I get the following error: None Exception “unhandled RuntimeError” Context has already been set File:/ /usr/lib/python3.5/multiprocessing/context.py, Line: 231

Roy

Van: Roy Lenders [mailto:roy.lenders@hotmail.com] Verzonden: maandag 4 juni 2018 9:29 Aan: 'ScottFreeLLC/AlphaPy' reply@reply.github.com Onderwerp: RE: [ScottFreeLLC/AlphaPy] Startup problems (#21)

Hello Mark,

My first issue was with pandas. It appears the library is very reliant on having the right version of Pandas (not the last one). I managed to solve this one by trying all Pandas versions between 1.19 and 2.22. When using 2.22 it worked.

The issue I now have is with installing tkinter within the Ubuntu environment. When I run marketflow, I get this tkinter issue. However, when I try to install tkinter for Python3, I get 404 errors from the Ubuntu archive.

Let’s hope I manage to solve these installation issues. One question about the library itself. I recently saw from academic research that AI models trained on Japanese stocks for example performed equally well on US stocks as compared to AI models trained on US stocks. So can I use Alphapy to train the algorithm on let’s say the history of 500 stocks and then use this complete trained model to predict on all invididual stocks? Instead of training only on the history of the specific stock you are trying to predict?

Roy

Van: Mark Conway [mailto:notifications@github.com] Verzonden: maandag 4 juni 2018 1:52 Aan: ScottFreeLLC/AlphaPy AlphaPy@noreply.github.com<mailto:AlphaPy@noreply.github.com> CC: roylend roy.lenders@hotmail.com<mailto:roy.lenders@hotmail.com>; Author author@noreply.github.com<mailto:author@noreply.github.com> Onderwerp: Re: [ScottFreeLLC/AlphaPy] Startup problems (#21)

Could you post some error logs here? Thanks.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/ScottFreeLLC/AlphaPy/issues/21#issuecomment-394201993, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AmCbhSEWEhTfevLARNHV9i2bARdcFnpIks5t5HahgaJpZM4UYM34.

roylend commented 6 years ago

So I encapsulated this mp.set_start_method(‘forkserver’) into a Try / except RunTimeError: phrase

Then Marketflow runs until a window pops up called “Client Forking” where I can choose with parent and child process. With both choices the running seems to be stop.

So the whole multi-thread process seems to be an issue somehow.

Roy

Van: Roy Lenders [mailto:roy.lenders@hotmail.com] Verzonden: maandag 4 juni 2018 10:14 Aan: 'ScottFreeLLC/AlphaPy' reply@reply.github.com Onderwerp: FW: [ScottFreeLLC/AlphaPy] Startup problems (#21)

Mark,

Managed to fix the tkinter issue as well by updating the URL’s for the Ubuntu environment to “old-releases”.

Now when I run marketflow or “main” in the directory with the Trading example. I get the following error: None Exception “unhandled RuntimeError” Context has already been set File:/ /usr/lib/python3.5/multiprocessing/context.py, Line: 231

Roy

Van: Roy Lenders [mailto:roy.lenders@hotmail.com] Verzonden: maandag 4 juni 2018 9:29 Aan: 'ScottFreeLLC/AlphaPy' reply@reply.github.com<mailto:reply@reply.github.com> Onderwerp: RE: [ScottFreeLLC/AlphaPy] Startup problems (#21)

Hello Mark,

My first issue was with pandas. It appears the library is very reliant on having the right version of Pandas (not the last one). I managed to solve this one by trying all Pandas versions between 1.19 and 2.22. When using 2.22 it worked.

The issue I now have is with installing tkinter within the Ubuntu environment. When I run marketflow, I get this tkinter issue. However, when I try to install tkinter for Python3, I get 404 errors from the Ubuntu archive.

Let’s hope I manage to solve these installation issues. One question about the library itself. I recently saw from academic research that AI models trained on Japanese stocks for example performed equally well on US stocks as compared to AI models trained on US stocks. So can I use Alphapy to train the algorithm on let’s say the history of 500 stocks and then use this complete trained model to predict on all invididual stocks? Instead of training only on the history of the specific stock you are trying to predict?

Roy

Van: Mark Conway [mailto:notifications@github.com] Verzonden: maandag 4 juni 2018 1:52 Aan: ScottFreeLLC/AlphaPy AlphaPy@noreply.github.com<mailto:AlphaPy@noreply.github.com> CC: roylend roy.lenders@hotmail.com<mailto:roy.lenders@hotmail.com>; Author author@noreply.github.com<mailto:author@noreply.github.com> Onderwerp: Re: [ScottFreeLLC/AlphaPy] Startup problems (#21)

Could you post some error logs here? Thanks.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/ScottFreeLLC/AlphaPy/issues/21#issuecomment-394201993, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AmCbhSEWEhTfevLARNHV9i2bARdcFnpIks5t5HahgaJpZM4UYM34.

roylend commented 6 years ago

I managed to completely run the example by commenting out two areas which generated bugs in Python3.5 in Ubuntu:

· Plot_learning_curve in plots.py

· Recursive feature elimation part in main.py

Roy

Van: Roy Lenders [mailto:roy.lenders@hotmail.com] Verzonden: maandag 4 juni 2018 11:30 Aan: 'ScottFreeLLC/AlphaPy' reply@reply.github.com Onderwerp: FW: [ScottFreeLLC/AlphaPy] Startup problems (#21)

So I encapsulated this mp.set_start_method(‘forkserver’) into a Try / except RunTimeError: phrase

Then Marketflow runs until a window pops up called “Client Forking” where I can choose with parent and child process. With both choices the running seems to be stop.

So the whole multi-thread process seems to be an issue somehow.

Roy

Van: Roy Lenders [mailto:roy.lenders@hotmail.com] Verzonden: maandag 4 juni 2018 10:14 Aan: 'ScottFreeLLC/AlphaPy' reply@reply.github.com<mailto:reply@reply.github.com> Onderwerp: FW: [ScottFreeLLC/AlphaPy] Startup problems (#21)

Mark,

Managed to fix the tkinter issue as well by updating the URL’s for the Ubuntu environment to “old-releases”.

Now when I run marketflow or “main” in the directory with the Trading example. I get the following error: None Exception “unhandled RuntimeError” Context has already been set File:/ /usr/lib/python3.5/multiprocessing/context.py, Line: 231

Roy

Van: Roy Lenders [mailto:roy.lenders@hotmail.com] Verzonden: maandag 4 juni 2018 9:29 Aan: 'ScottFreeLLC/AlphaPy' reply@reply.github.com<mailto:reply@reply.github.com> Onderwerp: RE: [ScottFreeLLC/AlphaPy] Startup problems (#21)

Hello Mark,

My first issue was with pandas. It appears the library is very reliant on having the right version of Pandas (not the last one). I managed to solve this one by trying all Pandas versions between 1.19 and 2.22. When using 2.22 it worked.

The issue I now have is with installing tkinter within the Ubuntu environment. When I run marketflow, I get this tkinter issue. However, when I try to install tkinter for Python3, I get 404 errors from the Ubuntu archive.

Let’s hope I manage to solve these installation issues. One question about the library itself. I recently saw from academic research that AI models trained on Japanese stocks for example performed equally well on US stocks as compared to AI models trained on US stocks. So can I use Alphapy to train the algorithm on let’s say the history of 500 stocks and then use this complete trained model to predict on all invididual stocks? Instead of training only on the history of the specific stock you are trying to predict?

Roy

Van: Mark Conway [mailto:notifications@github.com] Verzonden: maandag 4 juni 2018 1:52 Aan: ScottFreeLLC/AlphaPy AlphaPy@noreply.github.com<mailto:AlphaPy@noreply.github.com> CC: roylend roy.lenders@hotmail.com<mailto:roy.lenders@hotmail.com>; Author author@noreply.github.com<mailto:author@noreply.github.com> Onderwerp: Re: [ScottFreeLLC/AlphaPy] Startup problems (#21)

Could you post some error logs here? Thanks.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/ScottFreeLLC/AlphaPy/issues/21#issuecomment-394201993, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AmCbhSEWEhTfevLARNHV9i2bARdcFnpIks5t5HahgaJpZM4UYM34.

mrconway commented 6 years ago

Yes, the multithreading issue has been ongoing because of the njobs > 1 issues with xgboost, Keras, etc. I apologize for the examples not working; the package dependencies have been constantly breaking the code, specifically sklearn and pandas deprecations and API changes.

roylend commented 6 years ago

Mark,

I still any way to have multi-class classification modeled as well? I did notice two-class classification and regression in the two trading related examples but could not find how to use it in a multi-class environment.

Roy

Van: Mark Conway [mailto:notifications@github.com] Verzonden: maandag 4 juni 2018 17:21 Aan: ScottFreeLLC/AlphaPy AlphaPy@noreply.github.com CC: roylend roy.lenders@hotmail.com; Author author@noreply.github.com Onderwerp: Re: [ScottFreeLLC/AlphaPy] Startup problems (#21)

Yes, the multithreading issue has been ongoing because of the njobs > 1 issues with xgboost, Keras, etc. I apologize for the examples not working; the package dependencies have been constantly breaking the code, specifically sklearn and pandas deprecations and API changes.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/ScottFreeLLC/AlphaPy/issues/21#issuecomment-394393494, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AmCbhULyYsqrEQ-2VyKLgC0KmsCNCmtmks5t5VBJgaJpZM4UYM34.

roylend commented 6 years ago

Mark, we will be looking to also build an integration with Interactive Brokers API on top of this going forward? Has something like this already been build?

Roy

Van: Mark Conway [mailto:notifications@github.com] Verzonden: maandag 4 juni 2018 17:21 Aan: ScottFreeLLC/AlphaPy AlphaPy@noreply.github.com CC: roylend roy.lenders@hotmail.com; Author author@noreply.github.com Onderwerp: Re: [ScottFreeLLC/AlphaPy] Startup problems (#21)

Yes, the multithreading issue has been ongoing because of the njobs > 1 issues with xgboost, Keras, etc. I apologize for the examples not working; the package dependencies have been constantly breaking the code, specifically sklearn and pandas deprecations and API changes.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/ScottFreeLLC/AlphaPy/issues/21#issuecomment-394393494, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AmCbhULyYsqrEQ-2VyKLgC0KmsCNCmtmks5t5VBJgaJpZM4UYM34.

mrconway commented 5 years ago

Backend threading problems have been resolved.