M4 MLP is Linear Regression

[sjtrny]

From updater/models.py

        self.model = MLPRegressor(
            hidden_layer_sizes=6,
            activation="identity",
            solver="adam",
            max_iter=100,
            learning_rate="adaptive",
            learning_rate_init=0.001,
            random_state=42,
        )

The activation="identity" parameter means that no activation function is applied and this network can be replaced by a single identity layer.

This raises some questions:

• should we use a non-linear activation function (sigmoid, tanh or relu)?
• should we just replace this with linear regression?
• should we remove it entirely?

It takes a surprisingly long time to train these models so I think it is worthwhile to investigate alternatives.

[sjtrny]

This is even more important given that these models cause training to take hours. It's 1030 and the models are still running from the night before (0300 start).

[sjtrny]

Resolved by https://github.com/forecastlab/forecast_dash/pull/100