search

ITMO-NSS-team / fedot_electro_ts_case

The sources of the time-series forecasting real-world case with FEDOT.
https://github.com/aimclub/FEDOT
19 stars 8 forks source link

How to set loss function? #1

Closed Alexander-Minyushkin closed 3 years ago

Alexander-Minyushkin commented 3 years ago

I am trying to play with example https://github.com/ITMO-NSS-team/fedot_electro_ts_case/blob/main/case/traffic_example/simple_automl.ipynb

After execution of model = Fedot(problem='ts_forecasting', task_params=task.task_params)

chain = model.fit(features=train_data)

I see message: "Default loss function was set". Which one?

In file https://github.com/nccr-itmo/FEDOT/blob/master/fedot/api/main.py

We have default_test_metric_dict = { 'regression': ['rmse', 'mae'], 'classification': ['roc_auc', 'f1'], 'multiclassification': 'f1', 'clustering': 'silhouette', 'ts_forecasting': ['rmse', 'mae'] }

So it could be rmse or mae.

If I want to use rmse , how could I do that?

Dreamlone commented 3 years ago

Alexander, hi! We are glad you wanted to try FEDOT

About default_test_metric_dict - it is not a dictionary with metrics that are optimized. It is a dictionary with metrics values that will be counted when you call the get_metrics method. If you want to see the metrics that are used by default during optimization, please check this MetricByTask class.

As you can see, RMSE is used by default when solving the time series forecasting task

But if you want to use another metric, please use the following approach:

composer_params = {'max_depth': 3,
                   'max_arity': 4,
                   'pop_size': 20,
                   'num_of_generations': 20,
                   'learning_time': 1,
                   'preset': 'light_tun',
                   'metric': 'rmse'}
model = Fedot(problem='ts_forecasting', task_params=task.task_params, composer_params=composer_params)
chain = model.fit(features=train_data)

So, you can change metric 'rmse' to 'mae' or 'mape' for example

Good luck Feel free to ask more questions

J3FALL commented 3 years ago

@Alexander-Minyushkin thanks for feedback! Based on your issue we have merged minor changes with logging in API. Now it less shady, i think:

image