aimclub / FEDOT

Automated modeling and machine learning framework FEDOT
https://fedot.readthedocs.io
BSD 3-Clause "New" or "Revised" License
627 stars 86 forks source link

Bug with test_cv_api_correct #500

Closed valer1435 closed 2 years ago

valer1435 commented 2 years ago

Sometimes the test didn't work. I wrote a special test to reproduce the exception https://github.com/nccr-itmo/FEDOT/blob/617cc8dcd1e101ef36d1ab75703c758b1c7f9978/test/unit/validation/test_table_cv.py#L155

Use bug_test_cv_api_correct branch


..\..\..\fedot\core\pipelines\pipeline.py:179: in fit
    train_predicted = self._fit(input_data=copied_input_data,
..\..\..\fedot\core\pipelines\pipeline.py:147: in _fit
    train_predicted = self.root_node.fit(input_data=input_data)
..\..\..\fedot\core\pipelines\node.py:309: in fit
    secondary_input = self._input_from_parents(input_data=input_data, parent_operation='fit')
..\..\..\fedot\core\pipelines\node.py:336: in _input_from_parents
    parent_results, target = _combine_parents(parent_nodes, input_data,
..\..\..\fedot\core\pipelines\node.py:375: in _combine_parents
    prediction = parent.fit(input_data=input_data)
..\..\..\fedot\core\pipelines\node.py:309: in fit
    secondary_input = self._input_from_parents(input_data=input_data, parent_operation='fit')
..\..\..\fedot\core\pipelines\node.py:336: in _input_from_parents
    parent_results, target = _combine_parents(parent_nodes, input_data,
..\..\..\fedot\core\pipelines\node.py:375: in _combine_parents
    prediction = parent.fit(input_data=input_data)
..\..\..\fedot\core\pipelines\node.py:243: in fit
    return super().fit(input_data)
..\..\..\fedot\core\pipelines\node.py:161: in fit
    self.fitted_operation, operation_predict = self.operation.fit(params=self.content['params'],
..\..\..\fedot\core\operations\operation.py:83: in fit
    predict_train = self.predict(self.fitted_operation, data, is_fit_pipeline_stage, params)
..\..\..\fedot\core\operations\operation.py:104: in predict
    prediction = self._eval_strategy.predict(
..\..\..\fedot\core\operations\evaluation\classification.py:148: in predict
    prediction = trained_operation.transform(predict_data,
..\..\..\fedot\core\operations\evaluation\operation_implementations\data_operations\sklearn_imbalanced_class.py:84: in transform
    min_data, maj_data = self._get_data_by_target(features, target,
..\..\..\fedot\core\operations\evaluation\operation_implementations\data_operations\sklearn_imbalanced_class.py:125: in _get_data_by_target
    minority_data = np.hstack((features[min_idx], np.expand_dims(target[min_idx], 1)))
<__array_function__ internals>:5: in hstack
    ???
..\..\..\..\fedotenv_3_8\lib\site-packages\numpy\core\shape_base.py:346: in hstack
    return _nx.concatenate(arrs, 1)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

args = ([array([[ 0.95715102, 40.        ,  0.        , ...,  0.        ,
         0.        ,  1.        ],
       [ 0.65818...rray([[[0]],

       [[0]],

       [[0]],

       ...,

       [[0]],

       [[0]],

       [[0]]], dtype=int64)], 1)
kwargs = {}
relevant_args = [array([[ 0.95715102, 40.        ,  0.        , ...,  0.        ,
         0.        ,  1.        ],
       [ 0.658180...), array([[[0]],

       [[0]],

       [[0]],

       ...,

       [[0]],

       [[0]],

       [[0]]], dtype=int64)]

>   ???
E   ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 2 dimension(s) and the array at index 1 has 3 dimension(s)

<__array_function__ internals>:5: ValueError```
Dreamlone commented 2 years ago

It likely to be a problem with resampling operation

aPovidlo commented 2 years ago

@valer1435 @Dreamlone It seems like the target has the wrong shape with 2 dim. Each value is in the list. It is required to reshape it in 1 dim while preprocessing data.

Dreamlone commented 2 years ago

Was fixed in preprocessing PR