[BUG]: 'y_true' and 'y_pred' with just 1 value

[wasf84]

Description of the bug

Hi. First of all, thanks for that work. It's helping me a lot with my personal project.

I've noticed that when 'y_true' and 'y_pred' both has just 1 single value the code crashes.

I'm working on rainfall-runoff modeling to forecast few days ahead. When I try to forecast just 1 day ahead, it crashes during evaluation step.

Environment: Windows 11 Python 3.9.7 Permetrics 2.0.0

Thanks again for your attention.

Steps To Reproduce

import numpy as np from permetrics import RegressionMetric

y_true = np.array([3]) y_pred = np.array([2.5])

evaluator = RegressionMetric()

rmse_1 = evaluator.RMSE(y_true, y_pred) rmse_2 = evaluator.root_mean_squared_error(y_true, y_pred) print(f"RMSE: {rmse_1}, {rmse_2}")

mse = evaluator.MSE(y_true, y_pred) mae = evaluator.MAE(y_true, y_pred) print(f"MSE: {mse}, MAE: {mae}")

Additional Information

The output cell

{ "name": "IndexError", "message": "tuple index out of range", "stack": "--------------------------------------------------------------------------- IndexError Traceback (most recent call last) Cell In[237], line 10 7 evaluator = RegressionMetric() 9 ## 3.1 Call specific function inside object, each function has 2 names like below ---> 10 rmse_1 = evaluator.RMSE(y_true, y_pred) 11 rmse_2 = evaluator.root_mean_squared_error(y_true, y_pred) 12 print(f\"RMSE: {rmse_1}, {rmse_2}\")

File lib\site-packages\permetrics\regression.py:237, in RegressionMetric.root_mean_squared_error(self, y_true, y_pred, multi_output, force_finite, finite_value, kwargs) 222 def root_mean_squared_error(self, y_true=None, y_pred=None, multi_output=\"raw_values\", force_finite=True, finite_value=1.0, kwargs): 223 \"\"\" 224 Root Mean Squared Error (RMSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf) 225 (...) 235 result (float, int, np.ndarray): RMSE metric for single column or multiple columns 236 \"\"\" --> 237 y_true, y_pred, n_out = self.get_processed_data(y_true, y_pred) 238 result = np.sqrt(np.mean((y_true - y_pred) ** 2, axis=0)) 239 return self.get_output_result(result, n_out, multi_output, force_finite, finite_value=finite_value)

File lib\site-packages\permetrics\regression.py:119, in RegressionMetric.get_processed_data(self, y_true, y_pred, **kwargs) 108 \"\"\" 109 Args: 110 y_true (tuple, list, np.ndarray): The ground truth values (...) 116 n_out: Number of outputs 117 \"\"\" 118 if (y_true is not None) and (y_pred is not None): --> 119 y_true, y_pred, n_out = du.format_regression_data_type(y_true, y_pred) 120 else: 121 if (self.y_true is not None) and (self.y_pred is not None):

File lib\site-packages\permetrics\utils\data_util.py:22, in format_regression_data_type(y_true, y_pred) 20 if y_true.ndim > 2: 21 raise ValueError(\"y_true and y_pred must be 1D or 2D arrays.\") ---> 22 return y_true, y_pred, y_true.shape[1] # n_outputs 23 else: 24 raise ValueError(\"y_true and y_pred must have the same number of dimensions.\")

IndexError: tuple index out of range" }

[wasf84]

"I'm sorry. I forgot to mention that this error occurs with any of the metrics, not just the ones I mentioned here."

[thieu1995]

@wasf84, Yes, of course I will crash for any metrics. What is the point of using just 1 value to calculate metric? There are several metrics need to calculate the mean. With 1 value, you can't calculate the mean. I think you should get enough data before calculate metrics. If you forecast few days ahead. Then spend some more days to get data first then calculate its metrics later.

[wasf84]

Hi @thieu1995

I was trying to implement a Walf Forward Validation using this paper: https://www.sciencedirect.com/science/article/pii/S259012302400358X?via%3Dihub

I'm forecasting 1 day ahead, calculating the metrics, merging the results in a DataFrame, and so on, through the year of 2023. But I'll try to do what you said, getting more data and calculating the metrics at the end of experiment.

Thank you.