Closed Alexander-Minyushkin closed 3 years ago
Alexander, hi! We are glad you wanted to try FEDOT
About default_test_metric_dict - it is not a dictionary with metrics that are optimized. It is a dictionary with metrics values that will be counted when you call the get_metrics method. If you want to see the metrics that are used by default during optimization, please check this MetricByTask class.
As you can see, RMSE is used by default when solving the time series forecasting task
But if you want to use another metric, please use the following approach:
composer_params = {'max_depth': 3,
'max_arity': 4,
'pop_size': 20,
'num_of_generations': 20,
'learning_time': 1,
'preset': 'light_tun',
'metric': 'rmse'}
model = Fedot(problem='ts_forecasting', task_params=task.task_params, composer_params=composer_params)
chain = model.fit(features=train_data)
So, you can change metric 'rmse' to 'mae' or 'mape' for example
Good luck Feel free to ask more questions
@Alexander-Minyushkin thanks for feedback! Based on your issue we have merged minor changes with logging in API. Now it less shady, i think:
I am trying to play with example https://github.com/ITMO-NSS-team/fedot_electro_ts_case/blob/main/case/traffic_example/simple_automl.ipynb
After execution of model = Fedot(problem='ts_forecasting', task_params=task.task_params)
chain = model.fit(features=train_data)
I see message: "Default loss function was set". Which one?
In file https://github.com/nccr-itmo/FEDOT/blob/master/fedot/api/main.py
We have default_test_metric_dict = { 'regression': ['rmse', 'mae'], 'classification': ['roc_auc', 'f1'], 'multiclassification': 'f1', 'clustering': 'silhouette', 'ts_forecasting': ['rmse', 'mae'] }
So it could be rmse or mae.
If I want to use rmse , how could I do that?