Add Google Trends as a New Dataset

[slavakurilyak]

It is possible to quantify trading behavior in financial markets using Google Trends [1, 2, 3]. In other words, we can predict price movements of Bitcoin by analyzing Google search volume.

Inspecting Google Trends Search Volume vs Price of BTC/USD visually, a strong correlation appears.

Consider using GeneralMills's pytrends library to gather Google search volume data for "btc usd".

[slavakurilyak]

Consider creating a Google Trend strategy based on anyuzx's bitcoin-google-trend-strategy library with the following signals:

Version 1

When the relative change of google trend data for bitcoin is positive: BUY When the relative change of google trend data for bitcoin is negative: SELL

Version 2

When the relative change ratio, r, is positive: BUY using f(r) percentage of the available cash When the relative change ratio, r, is negive: SELL f(r) percentage of the available BTC

where "f(r) is a shifted sigmoid function with parameter scale."

Version 3

Version 3 is similar to version 2. But the parameters are separated in BULL and BEAR market. BULL/BEAR are determined by the derivative of the google trends data after applied a savgol fitering.

[slavakurilyak]

According to multiple sources, some scripting is necessary to gather daily Google Trends data.

According to anyuzx's bitcoin-google-trend-strategy notebook (Feb 2018), here are the limitations of Google Trends API:

First get google trend data. We want to get the day to day data But the google trend API only support monthly data if requesting the time frame longer than 6 month Thus we need to import by parts and combine them manually Here we import the google trend data from 2011-01-01 to 2018-01-09

According to t-davidson's google-trends-daily notebook (Feb 2017), here are the limitations of Google Trends API:

Google Trends only provides daily data for short time periods (the official limit is 90 days but it appears that daily values can be obtained for up to 9 months). During each period the data are scaled such that the period with the maximum volume of queries is set as 100 and other periods are given relative weights.

To join longer periods of data it is possible that we can get data from two overlapping periods and use the ratio between the observations from each of these periods to transform the rest of the series.

The problem is that Google does not release information about how the trends are calculated and it is unclear whether it is possible to use transformations to recover the daily trend.

According to clintonboys' trendy-scraper library (Feb 2017), here are the limitations of Google Trends API:

This script, written in Python, scrapes Google trends for a certain keyword over a long timescale. Since Google trends only outputs weekly data for periods longer than a few months, it also pulls daily data for small periods throughout the full period and then stitches them together, rescaling for consistency.

We first need to download all the bimonthly data.

Google usually returns daily data within a two month period, but if there is not a lot of data for a quarter it will return weekly data.

[treethought]

I'll look into these methods to for gathering daily trend data and implementing those signals. Here's a quick sample of monthly data provided by the pytrends library.