Machine Learning Based Indicators

[slavakurilyak]

User Story

As a machine learning developer, I want to use machine learning or deep learning to predict the direction of any crypto-asset (i.e. btc_usdt), so that I can develop ML-based indicators.

Consider

• Consider leveraging amicks's Speculator library, which currently supports:

1. Deep Neural Network via tensorflow library
2. Random Forest via scikit-learn library

Speculator can be deployed as a web app serving REST API prediction endpoints. For example: GET: /api/public/predict/?<bool:use_db>&<str:model_type>&<str:symbol>&<str:unit>&<int:count>&<int:period>&<int:partition>&<int:delta>&<int:seed>&<int:trees>&<int:jobs>&<DelimitedList<str>:longs>

• Consider leveraging maxim5's time-series-machine-learning library, which uses currently supports:

1. Ordinary linear model via numpy library
2. Gradient boosting xgboost library
3. Deep neural network via tensorflow library
4. Recurrent neural network: LSTM, GRU, one or multi-layered via tensorflow library
5. Convolutional neural network for 1-dimensional data via tensorflow library

• Consider using llens's CryptoCurrencyTrader library, which uses currently supports:

Support Vector Machine (SVM) Models

1. Epsilon-Support Vector Regression (SVR) via scikit-learn
2. C-Support Vector Classification (SVC) via scikit-learn

Ensemble Models

1. Random Forest Classifier via scikit-learn
2. Random Forest Regressor via scikit-learn
3. Ada Boost Classifier via scikit-learn
4. Ada Boost Regressor via scikit-learn
5. Gradient Boosting Classifier via scikit-learn
6. Gradient Boosting Regressor via scikit-learn
7. Extra Trees Classifier via scikit-learn
8. Extra Trees Regressor via scikit-learn

Recurrent Neural Network (RNN) Models

1. Long Short-Term Memory (LSTM) Models via tensorflow

• Consider using machine learning algorithms to learn indicators from unstructured data (Coates et al., 2011)

• Consider predicting the direction of cryptocurrency prices using deep learning algorithms, such as LSTM (Long Short Term Memory) (Ordóñez et at., 2016, Fischer et al., 2017, Pang et al., 2018).

Inspiration

• Askmike's gekko library, which is developing the integration of machine learning to replace technical analysis.

• For some research inspiration, check out Forecasting to Classification: Predicting the direction of stock market price using Xtreme Gradient Boosting

• @bukosabino's btctrading library which explores XGBoost on time-series data

[slavakurilyak]

@bukosabino Let's use XGBoost to predict the direction of btc_usdt for Bittrex or Poloniex.

[bukosabino]

I am working on this issue.

I have implemented it using an interface like catalyst library. So, I need to migrate to our way. How would we like to call this?

Option 1:

strat -ml xgboost

Option 2:

An algorithm in kryptos/platform/algos

[slavakurilyak]

I think we can create a strategy object triggered by $ strat -ml xgboost.

Once the strategy is working as expected, we can define the JSON config via xgboost.json

@treethought what do you think?

[treethought]

I think it'd be best to create a Strategy object, instead of a catalyst-style algo in platform/algos.

We probably want to implement some type of ML Indicator class so that xgboost and other ML indicators can be incorporated into any other Strategy object.

@bukosabino you can take a look at how TA indicators are implemented here. I'd be happy to help you integrate xgboost, so please reach out if you have any questions at all.

[bukosabino]

Hi all,

I have integrated 'xgboost' with our Strategy object interface and you can execute using:

$ strat -ml xgboost

So, now we can implement "AbstractIndicator" like a TAIndicator and MLIndicator. In the future, we will implement more ML models, such as LightGBM, LSTMs, etc.

I would like to continue working on this issue because I would like to add some features like:

• Write some extra utils results (confussion matrix).
• Add LightGBM (more efficient than xgboost).
• Add one graphic to show buy and sell possibilities.
• Apply ML models using TA like columns of dataframe. We could use in this way:

$ strat -ml xgboost -ta bbands

[treethought]

The integration looks great! nice job @bukosabino , excited to try it out

[bukosabino]

You need to use 'git pull' because I had some problems with Pipfile conflict. Now it is resolved.

[slavakurilyak]

Write some extra utils results (confussion matrix).

I think creating an extra utility to understand the confusion matrix is a great idea to better understand performance of a classification model. Calculating a confusion matrix can give us a better idea of what our classification model is getting right and what types of errors it is making. Confusion matrix can also help us in calculating more advanced classification metrics such as precision, recall, specificity and sensitivity of our classifier.

Add LightGBM (more efficient than xgboost).

Since we now have one boosting algorithm, we can put LightGBM on hold. If we need to improve the performance our solution, we can consider other's performance benmarks in comparison to various open source GBM implementations. We can also use Google's cloud infrastructure to ensure our machine learning models scale on the production server.

Add one graphic to show buy and sell possibilities.

Do you mean this?

Apply ML models using TA like columns of dataframe. We could use in this way: $ strat -ml xgboost -ta bbands

Check out task #64.

For additional machine learning tasks, check out task #25 and #65.