buy-and-hold run issues

[enriqueromualdez]

Hi all,

Ran into this issue when trying to run the buy_and_hold_backtest.py:

(qstraderp3) Enriques-MacBook-Air:examples enriqueromualdez$ python buy_and_hold_backtest.py Traceback (most recent call last): File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/pandas/init.py", line 25, in from pandas import hashtable, tslib, lib ImportError: dlopen(/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/pandas/lib.cpython-36m-darwin.so, 2): Symbol not found: _PySlice_AdjustIndices Referenced from: /Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/pandas/lib.cpython-36m-darwin.so Expected in: flat namespace in /Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/pandas/lib.cpython-36m-darwin.so

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "buy_and_hold_backtest.py", line 7, in from qstrader.trading_session import TradingSession File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/qstrader/trading_session.py", line 5, in from .price_handler.yahoo_daily_csv_bar import YahooDailyCsvBarPriceHandler File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/qstrader/price_handler/yahoo_daily_csv_bar.py", line 3, in import pandas as pd File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/pandas/init.py", line 31, in "extensions first.".format(module)) ImportError: C extension: dlopen(/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/pandas/lib.cpython-36m-darwin.so, 2): Symbol not found: _PySlice_AdjustIndices Referenced from: /Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/pandas/lib.cpython-36m-darwin.so Expected in: flat namespace in /Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/pandas/lib.cpython-36m-darwin.so not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace' to build the C extensions first.

Not sure where to go from here.. I did manually pip install yaml and munch to satisfy "no module" errors prior to this.

[enriqueromualdez]

Update:

I uninstalled then reinstalled via: pip install -r https://raw.githubusercontent.com/mhallsmoore/qstrader/master/requirements.txt

No issues with installation, but I've run into the same error once again. Any ideas folks?

[enriqueromualdez]

More info:

python --version; Python 3.6.0 :: Continuum Analytics, Inc.

[enriqueromualdez]

Update:

I was able to solve the issue by re-installing pandas using git clone. I then ran: python setup.py build_ext --inplace which seemed to solve the issue.. But now I'm faced with another bug.. yet again:

(qstraderp3) Enriques-MacBook-Air:examples enriqueromualdez$ python buy_and_hold_backtest.py Traceback (most recent call last): File "buy_and_hold_backtest.py", line 7, in from qstrader.trading_session import TradingSession File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/qstrader/trading_session.py", line 12, in from .statistics.tearsheet import TearsheetStatistics File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/qstrader/statistics/tearsheet.py", line 15, in import seaborn as sns File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/seaborn/init.py", line 6, in from .rcmod import * File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/seaborn/rcmod.py", line 8, in from . import palettes, _orig_rc_params File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/seaborn/palettes.py", line 12, in from .utils import desaturate, set_hls_values, get_color_cycle File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/seaborn/utils.py", line 15, in pandas_has_categoricals = LooseVersion(pd.version) >= "0.15" AttributeError: module 'pandas' has no attribute 'version'

I've uninstalled distutils, seaborn, and matplotlib, then reinstalled it, to no avail. I've also tried modifying pd.version to pd.version.version, but I believe this syntax was deprecated after pandas 0.15?

Quite frustrating. Hope someone can help out on this. @mhallsmoore @femtotrader

[enriqueromualdez]

Update:

Reinstalled everything once again. Backtest seems to be working, but now a new error...

While running buy_and_hold_backtest.py:

(qstraderp3) Enriques-MacBook-Air:examples enriqueromualdez$ python buy_and_hold_backtest.py Traceback (most recent call last): File "buy_and_hold_backtest.py", line 7, in from qstrader.trading_session import TradingSession File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/qstrader/trading_session.py", line 12, in from .statistics.tearsheet import TearsheetStatistics File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/qstrader/statistics/tearsheet.py", line 12, in import matplotlib.pyplot as plt File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/matplotlib/pyplot.py", line 114, in _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup() File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/matplotlib/backends/init.py", line 32, in pylab_setup globals(),locals(),[backend_name],0) File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/matplotlib/backends/backend_macosx.py", line 24, in from matplotlib.backends import _macosx RuntimeError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try one of the other backends. If you are Working with Matplotlib in a virtual enviroment see 'Working with Matplotlib in Virtual environments' in the Matplotlib FAQ (qstraderp3) Enriques-MacBook-Air:examples enriqueromualdez$ python buy_and_hold_backtest.py A configuration file named '~/qstrader.yml' is missing

Creating this file

---

CSV_DATA_DIR: ~/data OUTPUT_DIR: ~/out

You still have to create directories with data and put your data in!

Trying anyway with default configuration No tradelog files to clean. Running Backtest...

Backtest complete. Sharpe Ratio: 0.25 Max Drawdown: 79.59% Traceback (most recent call last): File "pandas/index.pyx", line 575, in pandas.index.DatetimeEngine.get_loc (pandas/index.c:10990) File "pandas/hashtable.pyx", line 303, in pandas.hashtable.Int64HashTable.get_item (pandas/hashtable.c:6589) TypeError: an integer is required

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/pandas/indexes/base.py", line 1945, in get_loc return self._engine.get_loc(key) File "pandas/index.pyx", line 538, in pandas.index.DatetimeEngine.get_loc (pandas/index.c:11140) File "pandas/index.pyx", line 577, in pandas.index.DatetimeEngine.get_loc (pandas/index.c:11046) File "pandas/index.pyx", line 583, in pandas.index.DatetimeEngine._date_check_type (pandas/index.c:11210) KeyError: 'next'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "pandas/index.pyx", line 575, in pandas.index.DatetimeEngine.get_loc (pandas/index.c:10990) File "pandas/hashtable.pyx", line 303, in pandas.hashtable.Int64HashTable.get_item (pandas/hashtable.c:6589) TypeError: an integer is required

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/pandas/tseries/index.py", line 1431, in get_loc return Index.get_loc(self, key, method, tolerance) File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/pandas/indexes/base.py", line 1947, in get_loc return self._engine.get_loc(self._maybe_cast_indexer(key)) File "pandas/index.pyx", line 538, in pandas.index.DatetimeEngine.get_loc (pandas/index.c:11140) File "pandas/index.pyx", line 577, in pandas.index.DatetimeEngine.get_loc (pandas/index.c:11046) File "pandas/index.pyx", line 583, in pandas.index.DatetimeEngine._date_check_type (pandas/index.c:11210) KeyError: 'next'

During handling of the above exception, another exception occurred:

ValueError: Error parsing datetime string "next" at position 0

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "buy_and_hold_backtest.py", line 70, in run(config, testing, tickers, filename) File "buy_and_hold_backtest.py", line 58, in run results = backtest.start_trading(testing=testing) File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/qstrader/trading_session.py", line 167, in start_trading self.statistics.plot_results() File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/qstrader/statistics/tearsheet.py", line 632, in plot_results self._plot_txt_curve(stats, ax=ax_txt_curve) File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/qstrader/statistics/tearsheet.py", line 356, in _plot_txt_curve sortino = perf.create_sortino_ratio(returns, self.periods) File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/qstrader/statistics/performance.py", line 66, in create_sortino_ratio return np.sqrt(periods) * (np.mean(returns)) / np.std(returns[returns < 0]) File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/pandas/core/series.py", line 618, in getitem if com.is_iterator(key): File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/pandas/core/common.py", line 1449, in is_iterator return hasattr(obj, 'next') or hasattr(obj, 'next') File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/pandas/core/generic.py", line 2670, in getattr if name in self._info_axis: File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/pandas/tseries/base.py", line 184, in contains res = self.get_loc(key) File "/Users/enriqueromualdez/venv/qstraderp3/lib/python3.6/site-packages/pandas/tseries/index.py", line 1439, in get_loc stamp = Timestamp(key, tz=self.tz) File "pandas/tslib.pyx", line 295, in pandas.tslib.Timestamp.new (pandas/tslib.c:9203) File "pandas/tslib.pyx", line 1269, in pandas.tslib.convert_to_tsobject (pandas/tslib.c:24653) File "pandas/tslib.pyx", line 1381, in pandas.tslib.convert_str_to_tsobject (pandas/tslib.c:26273) File "pandas/src/datetime.pxd", line 141, in datetime._string_to_dts (pandas/tslib.c:85631) SystemError: <class 'str'> returned a result with an error set

[mhallsmoore]

Hi,

Could you let me know which version of Pandas you're running?

Cheers,

Mike.

[enriqueromualdez]

Hi Mike,

Python3 pandas.version 0.19.2.

I actually got the backtest to run, but I’ve run into another issue. I’ve posted it on ticket #205.

Thanks,

Enrique

On May 31, 2017, at 5:30 PM, Michael Halls-Moore notifications@github.com wrote:

Hi,

Could you let me know which version of Pandas you're running?

Cheers,

Mike.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mhallsmoore/qstrader/issues/205#issuecomment-305137073, or mute the thread https://github.com/notifications/unsubscribe-auth/AbsmIv22BBlMVKVmQ_Lv82UWsS4EaYmeks5r_TMhgaJpZM4Npcv1.

[mhallsmoore]

Hi Enrique,

As it happens we've not yet built tests against Python 3.6 - only for Python 2.7, 3.4 and 3.5 I believe. It looks like this might in fact be a bug in Pandas itself when run in Python 3.6:

• https://github.com/pandas-dev/pandas/issues/14561
• https://stackoverflow.com/questions/42595429/how-to-convert-pd-series-to-datetime

Do you have access to a Python 3.5 distribution? That might help you get it working.

Kind regards,

Mike.

[enriqueromualdez]

Hey Mike,

Oh I see…Best then to uninstall Anaconda and re-install the 3.5 distribution?

Cheers,

Enrique

On May 31, 2017, at 5:46 PM, Michael Halls-Moore notifications@github.com wrote:

Hi Enrique,

As it happens we've not yet built tests against Python 3.6 - only for Python 2.7, 3.4 and 3.5 I believe. It looks like this might in fact be a bug in Pandas itself when run in Python 3.6:

pandas-dev/pandas#14561 https://github.com/pandas-dev/pandas/issues/14561 https://stackoverflow.com/questions/42595429/how-to-convert-pd-series-to-datetime https://stackoverflow.com/questions/42595429/how-to-convert-pd-series-to-datetime Do you have access to a Python 3.5 distribution? That might help you get it working.

Kind regards,

Mike.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mhallsmoore/qstrader/issues/205#issuecomment-305141109, or mute the thread https://github.com/notifications/unsubscribe-auth/AbsmIgHUUr3WkCsYLtvoNuIsH-fMN64Oks5r_Tb1gaJpZM4Npcv1.

[mhallsmoore]

Indeed, that's probably the best bet. If you still get some errors after that, post them here!

Also - just to let you know I think the Tearsheet has defaulted to being switched off when backtests are carried out, but it can be enabled quite easily. Let me know once you have a functioning backtest and I'll help you switch it on again!

[enriqueromualdez]

Much appreciated, Mike. I’ll give it a shot and get back to you!

On May 31, 2017, at 5:52 PM, Michael Halls-Moore notifications@github.com wrote:

Indeed, that's probably the best bet. If you still get some errors after that, post them here!

Also - just to let you know I think the Tearsheet has defaulted to being switched off when backtests are carried out, but it can be enabled quite easily. Let me know once you have a functioning backtest and I'll help you switch it on again!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mhallsmoore/qstrader/issues/205#issuecomment-305142569, or mute the thread https://github.com/notifications/unsubscribe-auth/AbsmIuSY7vtn4yRZiGY7ouXmY8H7mJzvks5r_ThTgaJpZM4Npcv1.

[enriqueromualdez]

Got it working! Read about the solution in a previous post: https://github.com/mhallsmoore/qstrader/issues/203

Some Mac users may also experience a bug with Matplotlib and it's backend settings in site-packages/matplotlib/matplotlibrc. A quick fix was to change the backend from macosx to TkAgg.

Thank you very much for your help, Mike! Looking forward to making use of QSTrader. Cheers!