Some showcase algorithms don't work

[kongehund]

Version: Latest binary distribution

As an example, trying to run Clenow's Stocks on the Move (v2) from TuringTrader results in the following output:

EXCEPTION: One or more errors occurred. (One or more errors occurred. (One or more errors occurred. (One or more errors occurred. (Failed to load data for Yahoo:BRK.B: One or more errors occurred. (Response status code does not indicate success: 404 (Not Found).)))))   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at TuringTrader.SimulatorV2.TimeSeries`1.get_Data() in C:\Users\fbert\OneDrive\Documents\TuringTrader\TuringTrader.Simulator\Simulator\v2\Core\TimeSeries.cs:line 222
   at TuringTrader.SimulatorV2.TimeSeries`1.get_Item(Int32 offset) in C:\Users\fbert\OneDrive\Documents\TuringTrader\TuringTrader.Simulator\Simulator\v2\Core\TimeSeries.cs:line 233
   at TuringTrader.BooksAndPubsV2.Clenow_StocksOnTheMove.<Run>b__51_1(String name)
   at System.Linq.EnumerableSorter`2.ComputeKeys(TElement[] elements, Int32 count)
   at System.Linq.EnumerableSorter`1.Sort(TElement[] elements, Int32 count)
   at System.Linq.OrderedEnumerable`1.ToList()
   at TuringTrader.BooksAndPubsV2.Clenow_StocksOnTheMove.<Run>b__51_0()
   at TuringTrader.SimulatorV2.Algorithm.<>c__DisplayClass71_0.<SimLoop>b__0() in C:\Users\fbert\OneDrive\Documents\TuringTrader\TuringTrader.Simulator\Simulator\v2\Core\Algorithm.cs:line 277
   at TuringTrader.SimulatorV2.Algorithm._simLoop(Func`2 innerBarFun, Double init) in C:\Users\fbert\OneDrive\Documents\TuringTrader\TuringTrader.Simulator\Simulator\v2\Core\Algorithm.cs:line 233
   at TuringTrader.SimulatorV2.Algorithm._simLoopOuter(Func`1 innerBarFun) in C:\Users\fbert\OneDrive\Documents\TuringTrader\TuringTrader.Simulator\Simulator\v2\Core\Algorithm.cs:line 247
   at TuringTrader.SimulatorV2.Algorithm.SimLoop(Action barFun) in C:\Users\fbert\OneDrive\Documents\TuringTrader\TuringTrader.Simulator\Simulator\v2\Core\Algorithm.cs:line 274
   at TuringTrader.BooksAndPubsV2.Clenow_StocksOnTheMove.Run()
   at TuringTrader.MainWindow.<>c__DisplayClass38_0.<RunButton_Click>g__uiThread|1() in C:\Users\fbert\OneDrive\Documents\TuringTrader\TuringTrader\MainWindow.xaml.cs:line 640
finished algorithm Clenow's Stocks on the Move after 1,2 seconds
EXCEPTION: Plotter: no data to plot   at TuringTrader.Simulator.Plotter.OpenWith(String pathToTemplateFile) in C:\Users\fbert\OneDrive\Documents\TuringTrader\TuringTrader.Simulator\Simulator\v1\Core\Plotter.cs:line 237
   at TuringTrader.SimulatorV2.Algorithm.Report() in C:\Users\fbert\OneDrive\Documents\TuringTrader\TuringTrader.Simulator\Simulator\v2\Core\Algorithm.cs:line 451
   at TuringTrader.MainWindow.<ReportButton_Click>b__39_0() in C:\Users\fbert\OneDrive\Documents\TuringTrader\TuringTrader\MainWindow.xaml.cs:line 693

[fbertram]

Hello Kongehund,

I am sorry to hear that you are facing difficulties. Unfortunately, I need more information to comment on the issue you are seeing.

At a first glance, this looks like TuringTrader failed to load one or more quotations from the data feed. Please let me know which data feed you are using. I suspect you might be using Yahoo Finance?

Further, it is worth pointing out that Clenow's Stocks on the Loose trades the whole S&P 500 universe. There are two problems with this:

(1) For once, Yahoo does not provide historical index constituents. As a workaround, TuringTrader includes static lists, which may become outdated (see https://github.com/fbertram/TuringTrader/blob/develop/TuringTrader.Simulator/Simulator/v2/Data/UniversesStatic.cs)

(2) Yahoo might try to deter users from downloading an excessive amount of quotes. I have seen them block my IP when attempting to download quotes for 500 stocks at once. I recommend changing this line (https://github.com/fbertram/TuringTrader/blob/5185a5c6755a7f2ffa29c2eac3cf1b8c54232bc5/BooksAndPubsV2/Clenow_StocksOnTheMove_v2.cs#L170) to load only five stocks and see if things are working then.

I am happy to resume this discussion, once I heard back from you regarding the data source you are using. However, it is worth pointing out that Yahoo really isn't a suitable data feed for running Clenow's strategy. At the very least, you should be switching to Tiingo instead.

Thank you, Best regards

Felix

[kongehund]

Hi Felix,

Thank you for the response!

I am currently in the learning stages of TuringTrader (and algorithmic trading in general), so I've been using Yahoo as my data source so far.

I have just tried Tiingo (free plan) and I now get a too many requests error. I understand that the basic plan only offers 50 requests per hour, and the Clenow algorithm does at least 500 requests (I assume a lot more, since the S&P 500 constituents change over time. I assume that I can fix this particular issue by either upgrading the Tiingo plan or, because previous requests are stored in cache, by running the algorithm many times with at least an hour between each time (as long as the simulation end date stays the same, to avoid requesting data for the latest hours or days). I appreciate to be corrected if I'm wrong about this assumption.

For reference, this is the too many requests error I get with Tiingo:

EXCEPTION: One or more errors occurred. (One or more errors occurred. (One or more errors occurred. (One or more errors occurred. (Failed to load meta for Tiingo:TMO: One or more errors occurred. (Response status code does not indicate success: 429 (Too Many Requests).)))))   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at TuringTrader.SimulatorV2.TimeSeries`1.get_Data() in C:\Users\fbert\OneDrive\Documents\TuringTrader\TuringTrader.Simulator\Simulator\v2\Core\TimeSeries.cs:line 222

[rest of stack omitted]

I am still interested in understanding what goes wrong when I use Yahoo as a data feed.

Best Regards, Kongehund

[fbertram]

Here is the relevant portion of the original error message you posted:

Failed to load data for Yahoo:BRK.B: One or more errors occurred. (Response status code does not indicate success: 404

As you can see Yahoo is not accepting 'BRK.B', and responding with a 404 error. I commited a fix for this to the development branch, which replaces '.' in tickers with '-', see https://github.com/fbertram/TuringTrader/commit/47447bc1daffacc6326dc90f3b7ccf4737565fa0). I am not sure when I will find the time to merge this back into the main branch and release a new official binary.

However, I'd like to point out that fixing this specific issue won't address a more serious problem: Neither Yahoo nor Tiingo offer historical index constituents. Using a static list for the S&P 500 constituents results in survivorship bias, which may skew the backtesting results.

In other words: If you are serious about algorithmic trading and trading individual stocks (not ETFs), you should subscribe to Norgate Data, which does provide historical index constituents to remedy this issue.

[kongehund]

Thank you for the explanation!

Since Norgate does not have data for European markets, I will look for a different feed as I'm from Denmark and NA markets have higher trading fees here.

[fbertram]

Hello Kongehund, for European markets, try Stooq as your data feed.

Best regards, Felix