ML with rule techniques

[vicmacbec]

Test a machine learning algorithm using trading rules as features.

[vicmacbec]

First feature importance.

Best half most importance pairs

[vicmacbec]

Looking the bias vs variance curves, we can see that there is overfitting since the test curve never down. To reduce the overfitting, we proceed to reduce the features using the most important half of features. We can observe that the the test curve does not down, so know we are going to proceed to use regularization.

[vicmacbec]

Changing the target of the increase of the 5 % of the price in the next week, the bias variance plot gives better solutions. The overfitting still remain but is more acceptable.

A view of a sample of the test orders is: where the red diamonds are false positive orders (maximice specificity), the black squares are false negative orders and the black squares with red diamonds are the true positive orders (maximice them)

Notes

• A specificity evaluation function was tried but it does not work, it might work with more test.
• Los mejores parámetros al momento son:
  • max_depth = 3,
  • eta = 0.001, (debe ser bajo por el overfitting, si es más bajo de 0.001 se necesitan más nrounds pero se acaba la RAM)
  • lambda = 10,
  • nrounds = 25000

[vicmacbec]

To do:

• Cross Validation
• Probar más iteraciones en Colab
• Grid Search
• Backtesting
• Re-entrenamiento
  • Si datos actuales ya no se ajustan al cumulative distribution function (cdf)
  • Si bajan el specificity con respecto al calculado en entrenamiento
  • Ejecución del modelo (multicore (uno por cada modelo))
  • Obtención de datos
  • Procesamiento de datos
  • Predicción de datos
  • Ejecución de compras/ventas

[vicmacbec]

Bias Variance plot using cross validation with 5 folds and parameters:

• max_depth = 3,
• eta = 0.001,
• lambda = 10,
• nrounds = 25000 We can observe how at the end of the iterations, the standard deviation is converging to the mean.

[vicmacbec]

• Grid search was done and the best parameters are:

• Execution of the model using the best parameters (the best test iteration is marked with a black point)

• Confussion matrix analysis

• AUC and ROC analysis

• Youden index computation

  • threshold: 0.6107571
  • specificity: 0.8219178
  • sensitivity: 0.7802198

• Confussion Matrix with Youden index: