Build DL model

[lorenzznerol]

• Which network is good for the given time series features?
• Should we deal with the features as if it was no time series, taking every week / month like a normal input feature?
• Get first baseline for DL and optimize hyper-parameters.
• Compare with previous ML baseline model

[lorenzznerol]

LSTM vs. Transformers

Since this is a time series, we should consider DL models which automatically find connections to past events as well since we building models with 100 features just to include the predecessors is not efficient on the long run (though possible).

LSTM is dead since it is only about a static look backwards, Transformers (attention-based) are the new trend to replace LSTM. LSTM might still be interesting for a prototype in which we already have little and aggregated data.

With Transformers, we might get around aggregation or clustering of the feature matrix as the preprocessing part and just directly train the model on the raw data instead. We could then search for anomalies and trends, and it is unclear to me what we will use as labels: the expensive kmeans, DBSCAN or DTW clustering of the input, or could we perhaps use Autoencoder as unsupervised learning in DL?

In Apache Spark: https://spark.apache.org/docs/1.4.0/api/scala/index.html#org.apache.spark.ml.Transformer

See Code Example: Estimator, Transformer, and Param at https://spark.apache.org/docs/1.4.0/ml-guide.html